Implement the NSGI bridge cdylib with an embedded CRuby VM#9
Merged
Conversation
Add the first working slice of the NSGI-compliant Ruby bridge: - ext/nsgi: cdylib exporting nsgi_handle/nsgi_free_response; boots an embedded CRuby VM on a dedicated thread (ruby_init_stack + ruby_setup + ruby_options + ruby_exec_node), hands off requests from foreign host threads via an MPSC queue with per-job condvars, wraps request fields as zero-copy read-only IO::Buffers (invalidated at request end), and serializes responses into a single app-owned malloc block freed by nsgi_free_response without VM interaction. Ruby exceptions, bridge raises, and Rust panics all degrade to a 500; nothing unwinds across the ABI. - lib: pure-Ruby side (NSGI.__dispatch, NSGI::Request) loaded by the embedded VM; normalizes app responses so the bridge needs no type checks. - host: mock NSGI host that dlopens the cdylib and exercises warmup, the exception path, buffer-lifetime enforcement (stash test), and 8x25 concurrent requests; runs clean under macOS leaks. - examples/hello.rb, gemspec, README, CLAUDE.md. Progress on #1 #2 #3 #6; upstream IO::Buffer gap filed as #8. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete the remaining roadmap items on the bridge: - Fiber-per-request concurrency (#4): the VM thread now runs a Ruby-side reactor (NSGI.__run) under NSGI::Scheduler, a minimal Fiber::Scheduler (io_wait, kernel_sleep, block/unblock with a thread-safe self-wake pipe, pluggable via NSGI.scheduler). Host threads enqueue jobs and wake the reactor through a pipe; each job runs in its own Fiber, so requests interleave whenever the app waits: 32 concurrent 50ms sleeps now finish in ~220ms instead of 1.6s. A crashed reactor degrades to serving static 500s instead of hanging host threads. - Lazy zero-copy request accessors (#2): NSGI::Request fields are created on first access via NSGI::Native callbacks, guarded by a live-request registry keyed by never-reused dispatch ids (job addresses suffer ABA reuse), so a stashed Request raises NSGI::Error instead of minting views over freed host memory. - IO::Buffer response bodies (#3): serialized straight from the backing memory, skipping the intermediate String copy. - Backpressure: NSGI_MAX_PENDING bounds the queue with a static 503. - SPEC.md (#7): the normative, implementation-independent app contract. - CI (#1, #6): build + mock host on Linux/macOS with valgrind memcheck and leaks; mock host grew stale-request, buffer-body, and wall-time-asserted fiber-interleaving coverage (239 requests, 0 leaks locally). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 7, 2026
Open
On Linux the cdylib links libruby.so.X.Y without recording its location, so a host dlopen'ing it fails with "cannot open shared object file" unless LD_LIBRARY_PATH happens to include Ruby's libdir (this broke CI; macOS only worked because Homebrew's libruby carries an absolute install_name). Emit -Wl,-rpath,<RbConfig libdir> from a build script, honoring the same RUBY interpreter override rb-sys uses. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mock host passed under valgrind but memcheck exited 1 with ~44k "conditional jump depends on uninitialised value" reports: CRuby's conservative GC scans raw (partly uninitialized) stack words by design, so this error class is pure noise when embedding the VM. Disable it with --undef-value-errors=no while keeping addressability checking, which is the class that would catch real bridge bugs (use-after-free, out-of-bounds arena writes, dangling request views). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First working slice of the NSGI-compliant Ruby bridge, covering the core of #1, #2, and #3, with initial verification tooling for #6.
nsgi_handle/nsgi_free_response. Boots an embedded CRuby VM on a dedicated thread using the canonical embedding sequence (ruby_init_stack+ruby_setup+ruby_options("-e", "0")+ruby_exec_node; bareruby_setup()leaves Ruby-defined core methods undefined, see the note on Set up the gem skeleton and boot an embedded CRuby VM inside the NSGI cdylib #1). Requests from foreign host threads are handed off via an MPSC queue with per-job condvars.IO::Bufferviews (rb_io_buffer_newwithEXTERNAL|READONLY); every buffer is tracked and force-invalidated when the request completes, on both success and raise paths.mallocblock ([NsgiHeader array][header bytes][body bytes]) whose base is recoverable from the struct itself, sonsgi_free_responsefrees it from any thread with zero VM interaction. A static, never-freed 500 is the last resort.NSGI.__dispatch; bridge-level raises are caught byrb_protect; Rust panics bycatch_unwind. Nothing unwinds across the ABI.NSGI::Request,NSGI.__dispatch); host/: mock NSGI host + test fixture; examples/hello.rb: minimal app.One upstream gap discovered during buffer-lifetime testing was filed separately as #8 (freed
IO::Bufferreads as empty instead of raising).Verification
The mock host dlopens the cdylib as a real host would and asserts: correct echo responses under 8-thread concurrency, app exception surfacing as a 500 (never a crash), a stashed buffer being unreadable after its request, and exactly-once
nsgi_free_responseon every path.🤖 Generated with Claude Code