IO::Buffer has no search primitives, forcing zero-copy consumers back into String copies to parse

[himura467]

Observed gap

IO::Buffer has no search primitives. The complete instance method list on current master (7c69f93d53):

& ^ <=> | ~ and! bit_count clear copy each each_byte empty? external? free
get_string get_value get_values hexdump initialize initialize_copy inspect
internal? lock locked locked? mapped? not! null? or! pread private? pwrite
read readonly? resize set_string set_value set_values shared? size slice
to_s transfer unlock valid? values write xor!

There is no #index, no delimiter scan, no substring search. Code that needs to find a byte or pattern inside a buffer has exactly two options:

1. get_string the region and use String#index and friends, which copies the entire region and defeats the purpose of holding an IO::Buffer in the first place.
2. Loop over get_value(:U8, offset) in Ruby, which is orders of magnitude slower than a memchr-class scan.

Why it matters for NSGI

NSGI exposes request fields to applications as zero-copy IO::Buffer views over host memory, so applications never pay for String allocation on data they do not touch. But the moment an application needs to parse inside a field (split a query string on & and =, find a \r\n\r\n or multipart boundary in a body, scan a header value for a delimiter), it is forced through option 1 above and copies the bytes anyway. The zero-copy pipeline ends exactly where real request processing begins.

Proposed upstream improvement

Add a search primitive to IO::Buffer, for example:

buffer.index(needle, offset = 0, length = size - offset) # => Integer or nil

where needle is an Integer byte, a String, or another IO::Buffer. Single-byte search can go straight to memchr (SIMD-accelerated in every mainstream libc, and the same routine String#index already benefits from); multi-byte search can use memmem where available. That one method is enough to build zero-copy tokenizers (query strings, header lists, line-based protocols) in pure Ruby, with each scan running at hardware speed instead of interpreter speed.

Possible follow-ups once #index exists, in the same spirit as the existing vectorizable #and!/#or!/#xor!/#bit_count family: #rindex, and an #each_until(delimiter) style iterator for framing.

Since IO::Buffer is explicitly experimental, extending the API surface here seems fair game, and it composes with the invalidation tightening proposed in #8.

References

• Zero-copy request views that hit this wall: lib/nsgi/request.rb, SPEC.md section 2
• Precedent for the technique: String#index (memchr), picohttpparser/httparse-style SIMD delimiter scanning in HTTP parsers