fix(gemini): emit reasoning chunks and fix thought parts extraction#56
Open
kirillreutski wants to merge 4 commits into
Open
fix(gemini): emit reasoning chunks and fix thought parts extraction#56kirillreutski wants to merge 4 commits into
kirillreutski wants to merge 4 commits into
Conversation
…ransport='provider'
Gemini API returns thinking content as parts with `thought: true`. Previously
these were pushed as regular text blocks regardless of reasoningTransport,
so think_chunk SSE events were never emitted for Gemini models.
When reasoningTransport is 'provider', parts with thought===true are now
pushed as { type: 'reasoning', reasoning } blocks. The agent emits these
as think_chunk_start / think_chunk / think_chunk_end progress events,
allowing clients to render thinking separately from the response text.
Default behavior (reasoningTransport: 'text') is unchanged — thought parts
continue to arrive as text blocks and get normalized by normalizeThinkBlocks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without includeThoughts, the model thinks internally but does not return thought parts in the response. This adds it automatically whenever thinkingBudget or thinkingLevel is set. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parseGeminiChunk now separates parts with thought===true into thoughtChunks when reasoningTransport is 'provider'. The streaming loop yields them as content_block_start(reasoning) + content_block_delta(reasoning_delta) before text chunks, so the agent emits think_chunk_start/think_chunk/think_chunk_end SSE events that clients can render separately from the response text. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When using
GeminiProviderwith thinking-capable models (e.g.gemini-2.5-pro), reasoning content is silently lost in three separate places:includeThoughtsmissing from request — Gemini API requiresincludeThoughts: trueinthinkingConfig, otherwise the model thinks internally but returns no thought parts in the response. The SDK was settingthinkingBudget/thinkingLevelwithout this flag.stream()drops thought parts —parseGeminiChunklumped all text parts intotextChunkswithout distinguishingpart.thought === true. Thought content was emitted as regular text instead ofthink_chunk_*SSE events.collectAllpath drops thought parts — The fallback path (when Gemini returns a JSON array instead of line-by-line SSE) had the same problem:thoughtChunkswere never extracted, so the agent never emitted reasoning events on this path either.A fourth issue: empty
part.textstrings ("") were being pushed intotextChunks, causing spurious emptytext_chunkSSE events.Changes
src/infra/providers/gemini.ts_buildGenerationConfig: addincludeThoughts: truewheneverthinkingBudgetorthinkingLevelis set.parseGeminiChunk: split parts bypart.thought === trueinto a newthoughtChunks: string[]return field (only whenreasoningTransport === 'provider'); skip empty strings.stream()— SSE path: consumethoughtChunksand yieldcontent_block_start(reasoning)+content_block_delta(reasoning_delta)events before text chunks.stream()—collectAllpath: same reasoning-chunk handling for the JSON-array fallback path.parseBlocks(used bycomplete()): mappart.thought === trueparts to{ type: 'reasoning', reasoning }blocks instead of text blocks.Testing
Verified end-to-end with
gemini-2.5-prousingreasoningTransport: 'provider':think_chunk_start/think_chunk/think_chunk_endSSE events now arrive correctly in streaming modecomplete()returns reasoning blocks in the message contentthoughtChunksstays empty)