Skip to content

keccak: benchmark ARM64 backends directly - #30

Draft
yperbasis wants to merge 1 commit into
masterfrom
review/arm64-backend-benchmark
Draft

keccak: benchmark ARM64 backends directly#30
yperbasis wants to merge 1 commit into
masterfrom
review/arm64-backend-benchmark

Conversation

@yperbasis

Copy link
Copy Markdown
Member

Summary

  • add an ARM64 benchmark that compares the SHA3 assembly and runtime fallback through the same streaming API
  • use Reset, Write, and Read with reusable output for both backends
  • avoid forcing SHA3 instructions when runtime detection did not select them

Why

Instruction availability does not prove that a backend is faster on every ARM core. This benchmark makes the dispatch choice measurable on Apple Silicon and Linux ARM64 systems. When SHA3 is unavailable, it reports the fallback only and never executes unsupported instructions.

This is a benchmark-only addition, so TDD is not applicable.

Benchmark

Apple M2 Max, Go 1.25.7, median of five 300 ms runs:

size       SHA3 assembly    x/crypto fallback
32 B          153.8 ns           352.5 ns
128 B         165.3 ns           351.3 ns
1 KiB        1182 ns             2678 ns
500 KiB       527.7 us           1.253 ms

Both paths report zero allocations with this call shape. The main value of the benchmark is running the same comparison on other ARM64 processors before changing dispatch policy.

Verification

  • go test ./...
  • go test -race ./...
  • go test -tags purego ./...
  • go vet ./...
  • go test -run '^$' -bench '^BenchmarkARM64Backends$' -benchmem -benchtime=300ms -count=5 ./...
  • go mod tidy -diff
  • golangci-lint run (twice)

}

func benchmarkARM64Backend(b *testing.B, native bool) {
for _, size := range []int{32, 128, 256, 1024, 4096, 500 * 1024} {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The size list duplicates benchSizes in keccak_test.go (identical literal), and strconv.Itoa(size) produces names that do not match the benchName scheme every other benchmark in this package uses.

Concrete effect: the output is BenchmarkARM64Backends/XCrypto/512000 while the existing fallback benchmark is BenchmarkXCrypto/500K. benchstat matches on the full benchmark name, so the new numbers cannot be lined up against the existing suite — which is the stated point of the PR ("running the same comparison on other ARM64 processors before changing dispatch policy").

Reuse the existing benchSizes var and benchName helper (both live in keccak_test.go, same package, no build-tag conflict).

@@ -0,0 +1,49 @@
//go:build arm64 && !purego

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing in this file is arm64-specific: useASM, Hasher, Reset/Write/Read all live in keccak_asm.go under (amd64 || arm64) && !purego, and keccak_amd64.go sets useASM = cpu.X86.HasBMI1 && cpu.X86.HasBMI2 — the exact same "is the asm backend actually faster than x/crypto on this core" question.

With the arm64 tag, an amd64 machine (the majority of Erigon nodes) cannot run this A/B at all, and a second near-identical file will be needed to answer the same question there. (amd64 || arm64) && !purego plus a neutral name (keccak_backend_bench_test.go / BenchmarkBackends) covers both with the same code.

var arm64BackendDigest [32]byte

func BenchmarkARM64Backends(b *testing.B) {
nativeAvailable := useASM

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

useASM is used here as a proxy for "the CPU has SHA3", but it is a mutable dispatch flag — and this very file is the precedent for mutating it. If any test or benchmark that runs earlier in the process leaves useASM == false (benchmarks run in file order, and a future keccak_a*_test.go would sort ahead of this one), then on a SHA3-capable machine:

  • nativeAvailable is false, so the SHA3 arm is silently dropped and the output shows only the fallback, with nothing saying why;
  • the deferred restore writes false back, so every later benchmark in the binary (BenchmarkFasterKeccak, BenchmarkFasterKeccakHasher, ...) measures x/crypto while claiming to measure the asm path.

Read the capability directly instead of the flag — runtime.GOOS == "darwin" || runtime.GOOS == "ios" || cpu.ARM64.HasSHA3, or factor that expression out of keccak_arm64.go into an immutable hasSHA3 var that init also uses.

b.Run(strconv.Itoa(size), func(b *testing.B) {
useASM = native
var h Hasher
h.Reset()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pre-loop h.Reset() is what makes the fallback arm report 0 allocs/op: with useASM == false, Hasher.Reset lazily allocates sha3.NewLegacyKeccak256(), and doing it before ResetTimer moves that allocation out of the measurement.

That is defensible for a steady-state comparison, but it is not visible in the output. Someone reading BenchmarkARM64Backends/XCrypto/512000 ... 0 allocs/op next to the existing BenchmarkXCrypto/500K ... 32 B/op 1 allocs/op will read it as the fallback having become allocation-free. Worth a one-line comment on the warm-up saying it exists to exclude the lazy xc construction.

h.Reset()
h.Write(data)
h.Read(out[:])
arm64BackendDigest = out

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arm64BackendDigest is only ever written, never read, so it works as an optimization barrier but nothing else. Since both arms already produce a digest of the same input, comparing them costs one package-level var and turns a silent divergence into a benchmark failure.

This is not hypothetical for a file whose whole job is flipping useASM: if a future change breaks one branch of the if !useASM ladder in Hasher.Read/Write (e.g. the fallback returns early, or sponge.Read stops padding), this benchmark keeps printing plausible ns/op for a backend that no longer hashes correctly. Record the digest on the native pass and b.Fatalf on mismatch in the fallback pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants