keccakf_amd64: drop NOSPLIT so the runtime can preempt the block loops - #32
keccakf_amd64: drop NOSPLIT so the runtime can preempt the block loops#32AskAlexSharov wants to merge 1 commit into
Conversation
keccakF1600BMI2 was NOSPLIT, which removes the stack check regardless of
frame size. That check is the only cooperative preemption point in the
absorb and squeeze loops that call it: the loops themselves call nothing
else, and assembly bodies are not async-preemptible, so the runtime has
nowhere to land. One large Sum256 or Write then holds every P in
stop-the-world for its whole duration.
Max GC stopping pause while hashing on an EPYC 4344P, GOMAXPROCS=2:
32 MiB 128 MiB
before 50.3 ms 67.1 ms
after 0.098 ms 0.016 ms
Throughput is unchanged: +0.25% geomean over -count=10, with 256 B and
128 B slightly faster and the large sizes about 1% slower.
The arm64 kernel never had NOSPLIT and does not show the problem, which is
what the comparison above rests on. The frame stays at 200 bytes: a leaf
with a zero frame gets no stack check either, so shrinking it would bring
the stall back.
Second machine (n0)Repeated on a second idle box, Max GC stopping pause while hashing,
The stall reproduces to the same bucket at 32 MiB on both boxes, and the fix lands in the tens-of-microseconds range on both. Throughput,
Same shape on both machines: the small sizes come out slightly faster, 1 KB and 4 KB slightly slower, and the total is a fraction of a percent. The one number that moved between boxes is 1 KB, +1.18% on n5 against +2.55% here; every size is ±0% within its own run, so that is a between-machine difference rather than measurement noise. n0 runs about 4% slower than n5 across the board on identical silicon, so treat the absolute figures as per-box and the deltas as the comparable part. |
1 participant
keccakF1600BMI2is declaredNOSPLIT, which drops the stack check whatever the frame size. That check is the only cooperative preemption point in the absorb and squeeze loops that call it — the loops call nothing else, and assembly bodies are not async-preemptible — so one largeSum256orWriteholds every P in stop-the-world until it returns.Max GC stopping pause while hashing, EPYC 4344P,
GOMAXPROCS=2:Throughput is unchanged —
-count=10, same idle box:arm64 never had
NOSPLITand does not show the stall, which is what the diagnosis rests on. The frame stays 200 bytes: a leaf with a zero frame gets no stack check either, so shrinking it reintroduces the stall — that is also why #12, which makes the arm64 kernelNOSPLIT, $0-16, would extend this behaviour to arm64. Measured there: 0.164 ms to 100.7 ms at 128 MiB.I first fixed this by chunking the Go-side block loops so they return to a function carrying a stack check every 16 KiB. That also works (0.049 ms) but costs 5.5-8.9% geomean and about 16% on 32 B inputs, and the cost stayed even when the small-input path executed code identical to master, so it is package layout rather than the added logic. Two lines beat it.
The generator carries the same change, so
go generatereproduces the committed.s, and a comment there records whyNOSPLITmust not come back.