Fix GPT OSS tool calls on Databricks - #124
Merged
Merged
Conversation
Databricks streams reasoning models (e.g. GPT OSS) with delta.content as
an array of reasoning/text parts instead of a string. The AI SDK rejects
the whole chunk, and Databricks sends tool call arguments in the same
chunk as reasoning, so tool calls completed with {} input.
Collapse array content to its text parts (or null) in the shared
OpenAI-compatible SSE transform, dropping reasoning parts.
The Unity AI Gateway's unified Responses API streams GPT OSS tool calls
with empty arguments in every event, response.completed included, so
every tool call reached the client as {} (or not at all). The same
request non-streamed, or streamed over gateway chat completions, carries
the arguments; Qwen and Llama stream them fine on Responses.
Withhold the mlflow-responses stamp from gpt-oss identities so they fall
back to openai-chat, which they advertise.
Keep output_text parts as well as text parts when collapsing array content, and test parts whose text is not a string. Note that gateway chat completions can also drop GPT OSS output on large requests, so it is the better route, not a clean one. Add Databricks to the fetch wrapper's provider list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix GPT OSS tool calls on Databricks.
Problem
GPT OSS models on Databricks lose their tool call arguments. The tool then runs with
{}and fails schema checks. There are two causes:delta.contentas an array of reasoning parts. The OpenAI spec says it must be a string or null. The AI SDK rejects the whole chunk, and the tool arguments are in that same chunk.arguments: ""in every event. Qwen and Llama stream their arguments correctly.Changes
openai-compat-fetch.ts: new transform 7. It turns arraycontentinto its text parts, ornull, and drops reasoning parts. The SDK already rejected array content for all providers, so this cannot break a provider that works today.databricks-helpers.ts:gpt-oss*models do not get themlflow-responsesprotocol. They useopenai-chat, which they also advertise.memory-bank/aiConfig.md: records the behaviour and the date we observed it.Known limits
Testing
streamText, and the GPT OSS routing rule. Both fail without the fix.databricks-gpt-oss-120bthrough the AI Gateway.