-
Notifications
You must be signed in to change notification settings - Fork 0
fix(chat): keep steering, action and injected messages in the conversation #2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: qa/agent-triggerdotdev-trigger-dev/pr-02-4816/base
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions can now become turns. `onAction` edits history with `chat.history`; to answer after the edit, return `chat.turn()` and a turn runs on the edited history with everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions, `onTurnStart` and `onTurnComplete`, and persistence. A regenerate is `chat.history.slice(0, -1); return chat.turn();`. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action }) => { | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| return chat.turn(); | ||
| } | ||
| if (action.type === "undo") chat.history.slice(0, -2); // edit only | ||
| }, | ||
| ``` | ||
|
|
||
| Returning a `StreamTextResult`, `string` or `UIMessage` from `onAction` is no longer supported and now fails with an error pointing to `chat.turn()`. A response produced that way skipped every turn guarantee, and its delivery to the browser was unreliable: the frontend never read the stream `transport.sendAction` returned, so a regenerate that appeared to work on the server did not render. | ||
|
|
||
| History edits made by an action are still persisted as before: platform-managed snapshots are written after the edit, and apps with their own store mirror the edit themselves. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages are now kept in the conversation when you drive turns yourself with `chat.createSession()` or `chat.MessageAccumulator`. Previously a message that arrived mid-answer shaped that answer and then existed nowhere: it was missing from `turn.uiMessages`, so an app persisting from there never stored it, missing from `turn.messages`, so every later turn answered as though it had never been sent, and it was not queued as its own turn either. It now lands in both, the same way it does on `chat.agent`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Injected system context is merged into a single instruction block, so it works on every supported AI SDK version. Note that a cached system prompt gives up its cache entry for as long as an injection is live, since the cached prefix has changed. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| `chat.inject()` with `role: "system"` now works. It previously put the system message into the conversation, which AI SDK 7 rejects for every provider: the next turn died with a generic "An error occurred." and persisted an empty assistant message, so the agent looked like it had stopped answering. System-role context is now appended to the model's instructions, which is also the only way to inject context the agent treats as trusted. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The changeset 'inject-system-to-instructions.md' states that system-role injections are delivered only via 'chat.toStreamTextOptions()', and a 'run()' that calls 'streamText' witho Impact: The changeset 'inject-system-to-instructions.md' states that system-role injections are delivered only via 'chat.toStreamTextOptions()', and a 'run()' that calls 'streamText' without spreading it silently drops the injection. This is a silent failure mode: the agent believes it has injected trusted context, but the model never sees it. The docs warn about this, but the SDK does not appear to throw or log when a syst… Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
|
|
||
| Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next turn only, rather than repeating on every turn that follows it. Every inference call in that turn sees it, so a `run()` that builds options more than once gets the same instructions each time. An instruction injected after an action has run, and before the next message, reaches that next turn rather than the one after it. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Undo, edit and regenerate now survive a run ending. History rolled back from `onAction` was only kept in the running worker's memory, so the rollback held while that worker stayed warm and then reverted on the next continuation. The undone messages came back, minutes later, with no error. This also holds when the turn before the action failed: the rollback used to be written against the cursor from before that turn, so a continuation could replay output the failed turn had already superseded. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages injected mid-answer are now part of the conversation, both for your hooks and for the model on later turns. Previously they reached the model for the answer they steered and reached the browser, but nothing else: `onTurnComplete` never saw them, so an app storing its own transcript lost the instruction the answer was shaped by, and it vanished from the conversation on reload. The model also forgot the instruction from the next turn onwards, answering as though the message had never been sent, while the chat UI still showed it. This holds when the steered turn fails part-way, and when `pendingMessages.prepare` reshapes the message: later turns now see the same form the steered turn did, not the original message. | ||
|
|
||
| Approving a tool call no longer undoes compaction. A tool-approval continuation used to rebuild the model's context from the full conversation, so a chat that had been summarised to fit the context window was sent the whole transcript again on the next call, and could go over the limit it had just been compacted to avoid. | ||
|
|
||
| If you worked around this by saving steering messages as they arrive, in `pendingMessages.onReceived` for example, that write now duplicates the one you get from `newUIMessages`. Drop it, or skip messages you have already stored. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions are sent through `useChat` so a turn that follows one renders like any turn. `TriggerChatTransport` recognises `body.action` on a `useChat` request and sends it as an action, so `sendMessage(undefined, { body: { action } })` or `regenerate({ body: { action } })` sends the action and `useChat` owns the response: it streams into the message list, `status` and `error` behave as for a message, and `stop` works. `useChatActions({ sendMessage })` in `@trigger.dev/sdk/chat/react` is a two-line convenience over that. | ||
|
|
||
| ```tsx | ||
| const { sendMessage } = useChat({ id: chatId, transport }); | ||
| const { sendAction } = useChatActions({ sendMessage }); | ||
| sendAction({ type: "regenerate" }); | ||
| ``` | ||
|
|
||
| Previously the frontend docs said `useChat` consumed the stream `transport.sendAction` returns; it never did, so an action's answer was never rendered by an app following them. `transport.sendAction` still returns a stream that callers outside `useChat` must read, and now accepts `{ abortSignal, metadata }`, with per-action metadata merged over the transport's `clientData`. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · LOW The changeset 'use-chat-actions.md' documents that 'transport.sendAction' now accepts '{ abortSignal, metadata }' with per-action metadata merged over the transport's 'clientData'. Impact: The changeset 'use-chat-actions.md' documents that 'transport.sendAction' now accepts '{ abortSignal, metadata }' with per-action metadata merged over the transport's 'clientData'. The diff does not show any validation or size limits on this metadata, and the changeset does not mention whether metadata is persisted, logged, or exposed to other tenants. If metadata is stored in the session snapshot or logs without sa… Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| --- | ||
| title: "Actions" | ||
| sidebarTitle: "Actions" | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn — undo, rollback, edit, regenerate." | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn: undo, rollback, edit, regenerate." | ||
| --- | ||
|
|
||
| ## Overview | ||
|
|
@@ -54,7 +54,7 @@ export const myChat = chat.agent({ | |
|
|
||
| ## Returning a model response from an action | ||
|
|
||
| `onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. The returned stream is auto-piped to the frontend just like a normal turn, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. | ||
| `onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. All three are sent to the frontend and added to the conversation just like a normal turn's answer, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. A returned `UIMessage` must have `role: "assistant"`; its text and `data-*` parts are delivered, and other part types are dropped. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The changeset for action-stream-into-conversation documents a breaking API change: returning StreamTextResult, string, or UIMessage from onAction now fails. Impact: The changeset for action-stream-into-conversation documents a breaking API change: returning StreamTextResult, string, or UIMessage from onAction now fails. The docs/ai-chat/actions.mdx still contains a section titled 'Returning a model response from an action' with code examples showing 'return streamText(...)' and 'return { role: "assistant", ... }'. If shipped together, users following the docs will hit the new r… Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages }) => { | ||
|
|
@@ -70,7 +70,37 @@ onAction: async ({ action, messages }) => { | |
| } | ||
| ``` | ||
|
|
||
| This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style). Persistence is your responsibility inside `onAction` itself; you have access to the streamed response object. | ||
| This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style). | ||
|
|
||
| ### Actions and persistence | ||
|
|
||
| An action is not a turn, so `onTurnComplete` never fires, and that is where an app that owns its own transcript normally writes. What that means depends on which persistence model you use. | ||
|
|
||
| **Platform-managed** (no `hydrateMessages`): nothing to do. After an action that changed the conversation (a `chat.history` mutation, a response returned from `onAction`, or both), the runtime writes the snapshot, so the change survives the run ending. | ||
|
|
||
| **Your own store** (`hydrateMessages` registered): the runtime deliberately does not write, because your store is the source of truth. A history mutation and a returned response both live only in the running worker until you persist them, and a continuation rehydrates from your store, not from what the worker had in memory. `chat.pipeAndCapture` hands you the same assistant message the runtime would have captured: | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages }) => { | ||
| if (action.type === "undo") { | ||
| chat.history.slice(0, -2); | ||
| await db.deleteLastExchange(chatId); // the rollback is yours to persist | ||
| } | ||
|
|
||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| await db.deleteLastAssistant(chatId); // drop the answer being replaced | ||
| const { message } = await chat.pipeAndCapture( | ||
| streamText({ model: anthropic("claude-sonnet-4-5"), messages }) | ||
| ); | ||
| if (message) await db.saveMessage(message); // then store the new one | ||
| } | ||
| }, | ||
| ``` | ||
|
|
||
| Mirror each mutation in your store, not only the additions. A `chat.history` mutation is invisible to your database, so a regenerate is a delete *and* an insert. Saving the new answer without removing the old one leaves both in the canonical transcript, and the next hydration returns the two of them. (An append-only or branching store is the exception: there you write a new version and resolve the head on read.) | ||
|
|
||
| Returning the stream instead of piping it yourself still works and still reaches the browser, but you have no message to store, so the next run does not know about it. | ||
|
|
||
| ## Gating actions on HITL state | ||
|
|
||
|
|
@@ -89,10 +119,10 @@ onAction: async ({ action, messages, signal }) => { | |
| ## Sending actions from the frontend | ||
|
|
||
| ```ts | ||
| // Browser — TriggerChatTransport | ||
| // Browser: TriggerChatTransport | ||
| const stream = await transport.sendAction(chatId, { type: "undo" }); | ||
|
|
||
| // Server — AgentChat | ||
| // Server: AgentChat | ||
| const stream = await agentChat.sendAction({ type: "rollback", targetMessageId: "msg-3" }); | ||
| ``` | ||
|
|
||
|
|
@@ -104,8 +134,8 @@ The action payload is validated against `actionSchema` on the backend; invalid a | |
|
|
||
| ## See also | ||
|
|
||
| - [`chat.history`](/ai-chat/backend#chat-history) — the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions) — `transport.sendAction` ergonomics | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) — fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations) — pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop) — gating fresh actions while a tool is waiting | ||
| - [`chat.history`](/ai-chat/backend#chat-history): the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions): `transport.sendAction` ergonomics | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages): fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations): pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop): gating fresh actions while a tool is waiting | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shipwright · HIGH
The new 'chat.turn()' API is introduced in the changeset but the diff does not show its implementation or type signature.
Impact: The new 'chat.turn()' API is introduced in the changeset but the diff does not show its implementation or type signature. The changeset example 'chat.history.slice(0, -1); return chat.turn();' relies on 'chat.history.slice' mutating in place, but the existing docs and tests use 'chat.history.set(...)' and 'chat.history.all()' for mutation. A new hire cannot tell whether 'slice' returns a new array (no-op) or mutates…
Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.