Skip to content

Pool worker fibers and coalesce reactor wake-ups#12

Merged
himura467 merged 1 commit into
mainfrom
perf-fiber-pool-wake-coalescing
Jul 7, 2026
Merged

Pool worker fibers and coalesce reactor wake-ups#12
himura467 merged 1 commit into
mainfrom
perf-fiber-pool-wake-coalescing

Conversation

@himura467

Copy link
Copy Markdown
Contributor

Summary

Two fixed-cost reductions motivated by the boundary benchmark (#11):

  • Fiber pooling: after finishing a request, a worker fiber parks itself back into the pool and is resumed with the next job, instead of being created and GC'd per request. Pushing before yielding is safe because the single-threaded VM cannot pop the fiber until the yield suspends it; dead fibers are skipped on pop.
  • Wake coalescing: nsgi_handle only writes a wake byte when it flips the wake flag false to true; the reactor resets the flag when a pop finds the queue empty. Both transitions happen under the queue mutex, so a producer that skips the write is guaranteed its job is seen by the drain loop the in-flight wake triggers (no lost-wakeup window).

Benchmark impact (Apple M2, 12-header baseline)

Metric Before After
8-thread throughput 161k req/s (6.2 us/req) 336k req/s (3.0 us/req)
Single-threaded 60k req/s (16.5 us/req) 99k req/s (10.1 us/req)
Allocations per request 12.0 10.0
Crossover vs idealized Rack ~60 headers ~22 headers

Scaling stays flat in header count and body size (60 headers: 339k req/s; 16 KB body: 338k req/s; 10 allocations in every scenario), which puts the crossover against the handoff-free idealized Rack harness inside the 15-25 header range of ordinary browser traffic. bench/README.md updated with the new matrix.

Verification

  • Mock host suite: 239 requests, all paths green, interleaving at 212ms.
  • leaks --atExit: 0 leaks.

🤖 Generated with Claude Code

Two fixed-cost reductions motivated by the boundary benchmark:

- Worker fibers are pooled: after finishing a request a fiber parks
  itself back into the pool and is resumed with the next job, instead of
  being created and GC'd per request. Pushing before yielding is safe
  because the single-threaded VM cannot pop the fiber until the yield
  suspends it; dead fibers are skipped on pop.
- Reactor wake-ups are coalesced: nsgi_handle only writes a wake byte
  when it flips the wake flag false to true, and the reactor resets the
  flag when a pop finds the queue empty. Both transitions happen under
  the queue mutex, so a producer that skips the write is guaranteed its
  job is seen by the drain loop the in-flight wake triggers.

Benchmark impact (Apple M2, 12-header baseline): 8-thread throughput
161k to 336k req/s (6.2 to 3.0 us/req), single-threaded 60k to 99k
req/s, allocations 12 to 10 per request. Scaling stays flat in header
count and body size, moving the crossover against the idealized
handoff-free Rack harness from ~60 request headers down to ~22, inside
the 15-25 range of ordinary browser traffic. Mock host suite and leaks
remain clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@himura467
himura467 merged commit e4d9419 into main Jul 7, 2026
2 checks passed
@himura467
himura467 deleted the perf-fiber-pool-wake-coalescing branch July 7, 2026 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant