fix(openai): keep streamed choices separate

[1fanwang]

What does this PR do?

OpenAI chat streams with n > 1 can interleave chunks from several choices. Langfuse currently combines them into one output, which corrupts both content and streamed tool arguments. After this change, each choice is recorded separately in choice.index order, while single-choice streams keep their existing output shape.

The stream parser now keeps one accumulator per choice. Tool-call indexes remain scoped to their choice, matching OpenAI's stream model. This completes the multiple-choice support introduced by #997, which covered request parameters and non-streaming responses.

Fixes langfuse/langfuse#16998

Type of change

• Bug fix
• New feature: not applicable
• Breaking change: not applicable
• Refactor: not applicable
• Documentation update: not applicable
• Tooling, CI, or repo maintenance: not applicable

Verification

uv run --frozen pytest -q tests/unit/test_openai.py::test_streaming_chat_completion_keeps_multiple_choices_separate

Raw logs

Before:

content: AB10
tool arguments: {"value":"B{"value":"A0"}1"}
expected content: ["A0", "B1"]

After:

test_streaming_chat_completion_keeps_multiple_choices_separate PASSED
1 passed in 2.11s

Checklist

• I self-reviewed the diff using code_review.md.
• I added or updated tests for behavior changes.
• I updated docs, examples, or .env.template if needed. No documentation changes are needed because this restores the existing n > 1 output contract for streams.
• I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
• I did not commit secrets or credentials.

Greptile Summary

This PR separates interleaved OpenAI streamed chat choices into per-choice accumulators while preserving the existing singleton shape for one observed choice.

• Keys chat accumulators by OpenAI choice index and returns multiple outputs in index order.
• Keeps tool-call indexes scoped to their respective choices.
• Adds a mocked streaming regression test covering interleaved content and tool arguments for n=2.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable changed-code failures identified.

The new accumulator isolates content and tool calls by choice index, preserves single-choice output behavior, and orders multi-choice telemetry consistently with the non-streaming response contract.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[OpenAI stream chunks] --> B[Read each choice index]
  B --> C[Per-choice accumulator]
  C --> D[Merge content and tool-call deltas]
  D --> E[Sort completed choices by index]
  E --> F{Number of observed choices}
  F -->|One| G[Singleton output]
  F -->|Multiple| H[Ordered output list]

Loading

_{Reviews (1): Last reviewed commit: "fix(openai): keep streamed choices separ..." | Re-trigger Greptile}

Context used:

• Knowledge Base — OpenAI instrumentation

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[CLAassistant]

All committers have signed the CLA.