Inflate HTTP/1.1 gzip and deflate without an EmbeddedChannel

[pavel-ptashyts]

Summary

Http1ContentDecompressor now inflates gzip, x-gzip, deflate and x-deflate itself instead of
through Netty's HttpContentDecompressor, which builds an EmbeddedChannel and a new Inflater for every
compressed response and copies every direct input buffer onto the heap. Every other encoding (br,
zstd, snappy, the Transfer-Encoding fallback) still goes through Netty exactly as before.

Closes #2356.

• An Inflater is borrowed from a small per-event-loop pool (FastThreadLocal, at most 16 idle per
  wrapper) for the length of one response and reset when it is given back. Idle inflaters are ended when
  the event-loop thread's thread locals are removed. Nothing is held per connection, so idle pooled
  connections keep no native zlib state.
• Input goes to the inflater as it is: the backing array for heap buffers, Inflater.setInput(ByteBuffer)
  (Java 11+) for direct ones; only a composite buffer with several components is copied.
• No EmbeddedChannel; decoded buffers are forwarded as DefaultHttpContent, cut at the same 64 KiB
  threshold (io.netty.compression.defaultMaxForwardBytes) as JdkZlibDecoder.
• maxDecompressedResponseSize is counted inside the inflate loop, across concatenated members, with the
  same DecompressionException message as before.
• Public API unchanged: same class, superclass and constructor (Revapi passes).

Compatibility

The gzip/deflate handling is a port of JdkZlibDecoder as HttpContentDecompressor configures it
(decompressConcatenated = true, ZLIB_OR_NONE for deflate, maxAllocation = 0): the same header and
trailer checks and messages, only the first magic byte checked, concatenated members, zlib-vs-raw detection,
a truncated stream ending quietly, fewer than 10 trailing bytes after a member ignored, output buffer growth
at the same points. Header rewriting, 100-continue pass-through and the read() requests issued with
autoRead off follow HttpContentDecoder, including reads that mix its messages with this handler's.

A differential test feeds Netty's HttpContentDecompressor and this handler the same bodies (33 bodies
including corrupt and truncated ones) cut into chunks in up to 7 ways, as heap, direct, pooled direct,
composite and read-only buffers, and requires the same headers, decoded bytes, number of decoded parts,
end of response, exception type and message, and that every buffer is released.

Deliberate differences:

• When a body turns out to be corrupt, Netty forwards what it had already decoded from that chunk and then
  throws; this handler throws without forwarding that partial chunk. The exchange fails with the same
  exception. As a side effect, with autoRead off a read is requested after the failing chunk where Netty
  might not request one.
• A handler further on that ends the response from inside a forward (removing this handler, or closing the
  channel, whose channelInactive Netty only queues) stops the decoding of that response; the rest of its
  body is not inflated and no LastHttpContent follows, so the exchange is completed by channelInactive.
  A body that ends with the connection still decodes its last part, which HttpClientCodec emits after the
  channel has gone inactive.
• -Dio.netty.noJdkZlibDecoder=true no longer switches gzip/deflate to JZlib, and a subclass overriding
  newContentDecoder(String) no longer sees these four encodings. Both are noted in the class Javadoc.

Known and left as is:

• A response handled by HttpContentDecompressor that never ends keeps Netty's EmbeddedChannel until the
  next response that Netty handles, or until the handler or channel goes away, because
  HttpContentDecoder.cleanup() is private. Such a response normally closes the connection.
• On threads that are not FastThreadLocalThreads (a custom EventLoopGroup thread factory) idle inflaters
  are freed by the JDK Cleaner rather than when the thread exits.
• Close and removal ordering is tested on EmbeddedChannel; on NIO/epoll isActive() also turns false
  inside close() and channelInactive is queued the same way.

Tests

Http1ContentDecompressorTest (28 tests): the differential test above; header rewriting and
keepEncodingHeader; full responses; trailers; 100-continue; Transfer-Encoding: gzip; alternating with
snappy (left to Netty) on one connection, including a Netty-handled response that never ends; the limit for
gzip, deflate and concatenated members; a capped allocator; the per-thread pool, its cap, reuse after a
removal and end() on thread-local removal; removal and close from a handler further on mid-chunk and while
the head is forwarded; close and removal mid-response; read requests with autoRead off. The existing
Http1DecompressionLimitTest is unchanged and passes.

Mutation checks during development: breaking the cross-chunk header path, the removal/close guards, the
activeAtDecode condition, the head check, the concatenated-member limit and end() in onRemoval each
fail a test. Removing the end-of-step buffer growth makes the inflate loop spin, so that path cannot be
dropped.

Verification

• ./mvnw clean verify -Dgpg.skip=true on OpenJDK 11.0.32 (Ubuntu, WSL2) at a19d7f27e: 1,819 tests,
  0 failures, 0 errors, 11 skipped; Error Prone, NullAway, Javadoc and Revapi pass. GPG signing was skipped
  because no key is available in that environment.

Benchmark

Http1ContentDecompressorBenchmark: one gzip response per operation on a keep-alive EmbeddedChannel,
JSON-like body, compressed input in 8 KiB direct pooled chunks, pooled allocator. JMH 1.37, OpenJDK
11.0.32, 2 forks, 5 x 2 s warmup, 10 x 2 s measurement, -prof gc. Allocation excludes pooled buffers on
both sides. The benchmark runs on JMH worker threads, so it takes FastThreadLocal's slower path; event
loops take the fast one.

Decoded body	Netty HttpContentDecompressor	Http1ContentDecompressor	Netty alloc	AHC alloc
2 KiB	6,854 ± 162 ns/op	5,251 ± 49 ns/op	2,910 B/op	1,160 B/op
20 KiB	23,838 ± 222 ns/op	21,597 ± 229 ns/op	5,538 B/op	1,160 B/op
200 KiB	285,662 ± 3,562 ns/op	276,711 ± 3,185 ns/op	33,527 B/op	2,435 B/op

Per response that is 1.6 to 9 us less CPU and 60 to 93% less allocation. The remaining ~1.2 KB/op is mostly
the benchmark's own response, header and content objects, which both sides allocate.

Review

Reviewed per the two-reviewer loop used by the author's team: Codex gpt-6.1-sol as guided reviewer and
Claude claude-sonnet-5-5 as independent reviewer, both at medium effort, read-only, a fresh session on
the current HEAD each round, models confirmed from the CLI output. Seven rounds; every actionable finding
was fixed or answered with the reasons listed above.

Round	Commit	Codex gpt-6.1-sol	Claude claude-sonnet-5-5
1	6d13083	1 major (removal during forward), 1 minor	minors (per-connection native memory, tests)
2	1485192	1 nit	minors (tests, comment)
3	67c668c	1 major (close during forward)	minors (buffer growth, tests)
4	0c9eb22	1 minor (growth point)	minors (close during head, input leak checks)
5	9c8a119	no findings	minors (FastThreadLocalThread coverage, cleanup)
6	bdd383d	1 minor (cleanup test too weak)	minors (cast guard)
7	a19d7f27e (now 39f64e6: only the commit message gained Closes #2356)	no findings	no blocker, major or minor; nits only

Round 7 nits not taken, with reasons: a 100-continue head without its LastHttpContent before a gzip
response cannot come out of HttpClientCodec; the inflater's reference to a released input is never read
before the next setInput()/reset(), and released unpooled direct memory is freed explicitly; the
comments state invariants the code cannot.

Claude Code on behalf of Pavel Ptashyts

🤖 Generated with Claude Code

[pavel-ptashyts]

CI failed on JDK 25 only (Linux, macOS, Windows), all in one test: idleInflatersAreEndedWithTheThreadLocals expected the NullPointerException an ended Inflater throws on JDK 11 to 21, while JDK 25 throws IllegalStateException. No production code was involved.

645506f makes the test accept any RuntimeException; an open inflater with no input throws nothing, so the test still proves the pooled inflater was ended. Checked on JDK 11 and JDK 25: it passes, and removing end() from onRemoval fails it on JDK 25 ("nothing was thrown"). ./mvnw clean verify -Dgpg.skip=true on JDK 11 at 645506f: 1,819 tests, 0 failures.

Review round 8 on 645506f, both at medium, read-only, also asked to look for anything else that could differ across JDK 11, 17, 21 and 25: Codex gpt-6.1-sol no findings; Claude claude-sonnet-5-5 no actionable findings. Remaining risk both name: read-only heap input to Inflater.setInput(ByteBuffer) on JDK 17 to 25 is covered by CI, not by reading the JDK sources.

Claude Code on behalf of Pavel Ptashyts