Skip to content

Commit 97f967e

Browse files
ericallamericallam
committed
Introduce the chat session API and better docs organization
1 parent e45533a commit 97f967e

File tree

11 files changed

+2236
-1413
lines changed

11 files changed

+2236
-1413
lines changed

‎docs/ai-chat/backend.mdx‎

Lines changed: 771 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/ai-chat/features.mdx‎

Lines changed: 421 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/ai-chat/frontend.mdx‎

Lines changed: 234 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,234 @@
1+
---
2+
title: "Frontend"
3+
sidebarTitle: "Frontend"
4+
description: "Transport setup, session management, client data, and frontend patterns for AI Chat."
5+
---
6+
7+
## Transport setup
8+
9+
Use the `useTriggerChatTransport` hook from `@trigger.dev/sdk/chat/react` to create a memoized transport instance, then pass it to `useChat`:
10+
11+
```tsx
12+
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";
13+
import { useChat } from "@ai-sdk/react";
14+
import type { myChat } from "@/trigger/chat";
15+
import { getChatToken } from "@/app/actions";
16+
17+
export function Chat() {
18+
const transport = useTriggerChatTransport<typeof myChat>({
19+
task: "my-chat",
20+
accessToken: getChatToken,
21+
});
22+
23+
const { messages, sendMessage, stop, status } = useChat({ transport });
24+
// ... render UI
25+
}
26+
```
27+
28+
The transport is created once on first render and reused across re-renders. Pass a type parameter for compile-time validation of the task ID.
29+
30+
<Tip>
31+
The hook keeps `onSessionChange` up to date via a ref internally, so you don't need to memoize the callback or worry about stale closures.
32+
</Tip>
33+
34+
### Dynamic access tokens
35+
36+
For token refresh, pass a function instead of a string. It's called on each `sendMessage`:
37+
38+
```ts
39+
const transport = useTriggerChatTransport({
40+
task: "my-chat",
41+
accessToken: async () => {
42+
const res = await fetch("/api/chat-token");
43+
return res.text();
44+
},
45+
});
46+
```
47+
48+
## Session management
49+
50+
### Session cleanup (frontend)
51+
52+
Since session creation and updates are handled server-side, the frontend only needs to handle session deletion when a run ends:
53+
54+
```tsx
55+
const transport = useTriggerChatTransport<typeof myChat>({
56+
task: "my-chat",
57+
accessToken: getChatToken,
58+
sessions: loadedSessions, // Restored from DB on page load
59+
onSessionChange: (chatId, session) => {
60+
if (!session) {
61+
deleteSession(chatId); // Server action — run ended
62+
}
63+
},
64+
});
65+
```
66+
67+
### Restoring on page load
68+
69+
On page load, fetch both the messages and the session from your database, then pass them to `useChat` and the transport. Pass `resume: true` to `useChat` when there's an existing conversation — this tells the AI SDK to reconnect to the stream via the transport.
70+
71+
```tsx app/page.tsx
72+
"use client";
73+
74+
import { useEffect, useState } from "react";
75+
import { useTriggerChatTransport } from "@trigger.dev/sdk/chat/react";
76+
import { useChat } from "@ai-sdk/react";
77+
import { getChatToken, getChatMessages, getSession, deleteSession } from "@/app/actions";
78+
79+
export default function ChatPage({ chatId }: { chatId: string }) {
80+
const [initialMessages, setInitialMessages] = useState([]);
81+
const [initialSession, setInitialSession] = useState(undefined);
82+
const [loaded, setLoaded] = useState(false);
83+
84+
useEffect(() => {
85+
async function load() {
86+
const [messages, session] = await Promise.all([
87+
getChatMessages(chatId),
88+
getSession(chatId),
89+
]);
90+
setInitialMessages(messages);
91+
setInitialSession(session ? { [chatId]: session } : undefined);
92+
setLoaded(true);
93+
}
94+
load();
95+
}, [chatId]);
96+
97+
if (!loaded) return null;
98+
99+
return (
100+
<ChatClient
101+
chatId={chatId}
102+
initialMessages={initialMessages}
103+
initialSessions={initialSession}
104+
/>
105+
);
106+
}
107+
108+
function ChatClient({ chatId, initialMessages, initialSessions }) {
109+
const transport = useTriggerChatTransport({
110+
task: "my-chat",
111+
accessToken: getChatToken,
112+
sessions: initialSessions,
113+
onSessionChange: (id, session) => {
114+
if (!session) deleteSession(id);
115+
},
116+
});
117+
118+
const { messages, sendMessage, stop, status } = useChat({
119+
id: chatId,
120+
messages: initialMessages,
121+
transport,
122+
resume: initialMessages.length > 0, // Resume if there's an existing conversation
123+
});
124+
125+
// ... render UI
126+
}
127+
```
128+
129+
<Info>
130+
`resume: true` causes `useChat` to call `reconnectToStream` on the transport when the component mounts. The transport uses the session's `lastEventId` to skip past already-seen stream events, so the frontend only receives new data. Only enable `resume` when there are existing messages — for brand new chats, there's nothing to reconnect to.
131+
</Info>
132+
133+
<Warning>
134+
In React strict mode (enabled by default in Next.js dev), you may see a `TypeError: Cannot read properties of undefined (reading 'state')` in the console when using `resume`. This is a [known bug in the AI SDK](https://github.com/vercel/ai/issues/8477) caused by React strict mode double-firing the resume effect. The error is caught internally and **does not affect functionality** — streaming and message display work correctly. It only appears in development and will not occur in production builds.
135+
</Warning>
136+
137+
## Client data and metadata
138+
139+
### Transport-level client data
140+
141+
Set default client data on the transport that's included in every request. When the task uses `clientDataSchema`, this is type-checked to match:
142+
143+
```ts
144+
const transport = useTriggerChatTransport<typeof myChat>({
145+
task: "my-chat",
146+
accessToken: getChatToken,
147+
clientData: { userId: currentUser.id },
148+
});
149+
```
150+
151+
### Per-message metadata
152+
153+
Pass metadata with individual messages via `sendMessage`. Per-message values are merged with transport-level client data (per-message wins on conflicts):
154+
155+
```ts
156+
sendMessage(
157+
{ text: "Hello" },
158+
{ metadata: { model: "gpt-4o", priority: "high" } }
159+
);
160+
```
161+
162+
### Typed client data with clientDataSchema
163+
164+
Instead of manually parsing `clientData` with Zod in every hook, pass a `clientDataSchema` to `chat.task`. The schema validates the data once per turn, and `clientData` is typed in all hooks and `run`:
165+
166+
```ts
167+
import { chat } from "@trigger.dev/sdk/ai";
168+
import { streamText } from "ai";
169+
import { openai } from "@ai-sdk/openai";
170+
import { z } from "zod";
171+
172+
export const myChat = chat.task({
173+
id: "my-chat",
174+
clientDataSchema: z.object({
175+
model: z.string().optional(),
176+
userId: z.string(),
177+
}),
178+
onChatStart: async ({ chatId, clientData }) => {
179+
// clientData is typed as { model?: string; userId: string }
180+
await db.chat.create({
181+
data: { id: chatId, userId: clientData.userId },
182+
});
183+
},
184+
run: async ({ messages, clientData, signal }) => {
185+
// Same typed clientData — no manual parsing needed
186+
return streamText({
187+
model: openai(clientData?.model ?? "gpt-4o"),
188+
messages,
189+
abortSignal: signal,
190+
});
191+
},
192+
});
193+
```
194+
195+
The schema also types the `clientData` option on the frontend transport:
196+
197+
```ts
198+
// TypeScript enforces that clientData matches the schema
199+
const transport = useTriggerChatTransport<typeof myChat>({
200+
task: "my-chat",
201+
accessToken: getChatToken,
202+
clientData: { userId: currentUser.id },
203+
});
204+
```
205+
206+
Supports Zod, ArkType, Valibot, and other schema libraries supported by the SDK.
207+
208+
## Stop generation
209+
210+
Calling `stop()` from `useChat` sends a stop signal to the running task via input streams. The task aborts the current `streamText` call, but the run stays alive for the next message:
211+
212+
```tsx
213+
const { messages, sendMessage, stop, status } = useChat({ transport });
214+
215+
{status === "streaming" && (
216+
<button type="button" onClick={stop}>
217+
Stop
218+
</button>
219+
)}
220+
```
221+
222+
See [Stop generation](/ai-chat/backend#stop-generation) in the backend docs for how to handle stop signals in your task.
223+
224+
## Self-hosting
225+
226+
If you're self-hosting Trigger.dev, pass the `baseURL` option:
227+
228+
```ts
229+
const transport = useTriggerChatTransport({
230+
task: "my-chat",
231+
accessToken,
232+
baseURL: "https://your-trigger-instance.com",
233+
});
234+
```

‎docs/ai-chat/overview.mdx‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
---
2+
title: "AI Chat"
3+
sidebarTitle: "Overview"
4+
description: "Run AI SDK chat completions as durable Trigger.dev tasks with built-in realtime streaming, multi-turn conversations, and message persistence."
5+
---
6+
7+
## Overview
8+
9+
The `@trigger.dev/sdk` provides a custom [ChatTransport](https://sdk.vercel.ai/docs/ai-sdk-ui/transport) for the Vercel AI SDK's `useChat` hook. This lets you run chat completions as **durable Trigger.dev tasks** instead of fragile API routes — with automatic retries, observability, and realtime streaming built in.
10+
11+
**How it works:**
12+
1. The frontend sends messages via `useChat` through `TriggerChatTransport`
13+
2. The first message triggers a Trigger.dev task; subsequent messages resume the **same run** via input streams
14+
3. The task streams `UIMessageChunk` events back via Trigger.dev's realtime streams
15+
4. The AI SDK's `useChat` processes the stream natively — text, tool calls, reasoning, etc.
16+
5. Between turns, the run stays warm briefly then suspends (freeing compute) until the next message
17+
18+
No custom API routes needed. Your chat backend is a Trigger.dev task.
19+
20+
<Accordion title="How it works (sequence diagrams)">
21+
22+
### First message flow
23+
24+
```mermaid
25+
sequenceDiagram
26+
participant User
27+
participant useChat as useChat + Transport
28+
participant API as Trigger.dev API
29+
participant Task as chat.task Worker
30+
participant LLM as LLM Provider
31+
32+
User->>useChat: sendMessage("Hello")
33+
useChat->>useChat: No session for chatId → trigger new run
34+
useChat->>API: triggerTask(payload, tags: [chat:id])
35+
API-->>useChat: { runId, publicAccessToken }
36+
useChat->>useChat: Store session, subscribe to SSE
37+
38+
API->>Task: Start run with ChatTaskWirePayload
39+
Task->>Task: onChatStart({ chatId, messages, clientData })
40+
Task->>Task: onTurnStart({ chatId, messages })
41+
Task->>LLM: streamText({ model, messages, abortSignal })
42+
LLM-->>Task: Stream response chunks
43+
Task->>API: streams.pipe("chat", uiStream)
44+
API-->>useChat: SSE: UIMessageChunks
45+
useChat-->>User: Render streaming text
46+
Task->>API: Write __trigger_turn_complete
47+
API-->>useChat: SSE: turn complete + refreshed token
48+
useChat->>useChat: Close stream, update session
49+
Task->>Task: onTurnComplete({ messages, stopped: false })
50+
Task->>Task: Wait for next message (warm → suspend)
51+
```
52+
53+
### Multi-turn flow
54+
55+
```mermaid
56+
sequenceDiagram
57+
participant User
58+
participant useChat as useChat + Transport
59+
participant API as Trigger.dev API
60+
participant Task as chat.task Worker
61+
participant LLM as LLM Provider
62+
63+
Note over Task: Suspended, waiting for message
64+
65+
User->>useChat: sendMessage("Tell me more")
66+
useChat->>useChat: Session exists → send via input stream
67+
useChat->>API: sendInputStream(runId, "chat-messages", payload)
68+
Note right of useChat: Only sends new message (not full history)
69+
70+
API->>Task: Deliver to messagesInput
71+
Task->>Task: Wake from suspend
72+
Task->>Task: Append to accumulated messages
73+
Task->>Task: onTurnStart({ turn: 1 })
74+
Task->>LLM: streamText({ messages: [all accumulated] })
75+
LLM-->>Task: Stream response
76+
Task->>API: streams.pipe("chat", uiStream)
77+
API-->>useChat: SSE: UIMessageChunks
78+
useChat-->>User: Render streaming text
79+
Task->>API: Write __trigger_turn_complete
80+
Task->>Task: onTurnComplete({ turn: 1 })
81+
Task->>Task: Wait for next message (warm → suspend)
82+
```
83+
84+
### Stop signal flow
85+
86+
```mermaid
87+
sequenceDiagram
88+
participant User
89+
participant useChat as useChat + Transport
90+
participant API as Trigger.dev API
91+
participant Task as chat.task Worker
92+
participant LLM as LLM Provider
93+
94+
Note over Task: Streaming response...
95+
96+
User->>useChat: Click "Stop"
97+
useChat->>API: sendInputStream(runId, "chat-stop", { stop: true })
98+
API->>Task: Deliver to stopInput
99+
Task->>Task: stopController.abort()
100+
LLM-->>Task: Stream ends (AbortError)
101+
Task->>Task: cleanupAbortedParts(responseMessage)
102+
Note right of Task: Remove partial tool calls,<br/>mark streaming parts as done
103+
Task->>API: Write __trigger_turn_complete
104+
API-->>useChat: SSE: turn complete
105+
Task->>Task: onTurnComplete({ stopped: true })
106+
Task->>Task: Wait for next message
107+
```
108+
109+
</Accordion>
110+
111+
<Note>
112+
Requires `@trigger.dev/sdk` version **4.4.0 or later** and the `ai` package **v5.0.0 or later**.
113+
</Note>
114+
115+
## How multi-turn works
116+
117+
### One run, many turns
118+
119+
The entire conversation lives in a **single Trigger.dev run**. After each AI response, the run waits for the next message via input streams. The frontend transport handles this automatically — it triggers a new run for the first message, and sends subsequent messages to the existing run.
120+
121+
This means your conversation has full observability in the Trigger.dev dashboard: every turn is a span inside the same run.
122+
123+
### Warm and suspended states
124+
125+
After each turn, the run goes through two phases of waiting:
126+
127+
1. **Warm phase** (default 30s) — The run stays active and responds instantly to the next message. Uses compute.
128+
2. **Suspended phase** (default up to 1h) — The run suspends, freeing compute. It wakes when the next message arrives. There's a brief delay as the run resumes.
129+
130+
If no message arrives within the turn timeout, the run ends gracefully. The next message from the frontend will automatically start a fresh run.
131+
132+
<Info>
133+
You are not charged for compute during the suspended phase. Only the warm phase uses compute resources.
134+
</Info>
135+
136+
### What the backend accumulates
137+
138+
The backend automatically accumulates the full conversation history across turns. After the first turn, the frontend transport only sends the new user message — not the entire history. This is handled transparently by the transport and task.
139+
140+
The accumulated messages are available in:
141+
- `run()` as `messages` (`ModelMessage[]`) — for passing to `streamText`
142+
- `onTurnStart()` as `uiMessages` (`UIMessage[]`) — for persisting before streaming
143+
- `onTurnComplete()` as `uiMessages` (`UIMessage[]`) — for persisting after the response
144+
145+
## Three approaches
146+
147+
There are three ways to build the backend, from most opinionated to most flexible:
148+
149+
| Approach | Use when | What you get |
150+
|----------|----------|--------------|
151+
| [chat.task()](/ai-chat/backend#chattask) | Most apps | Auto-piping, lifecycle hooks, message accumulation, stop handling |
152+
| [chat.createSession()](/ai-chat/backend#chatcreatesession) | Need a loop but not hooks | Async iterator with per-turn helpers, message accumulation, stop handling |
153+
| [Raw task + primitives](/ai-chat/backend#raw-task-with-primitives) | Full control | Manual control of every step — use `chat.messages`, `chat.createStopSignal()`, etc. |
154+
155+
## Related
156+
157+
- [Quick Start](/ai-chat/quick-start) — Get a working chat in 3 steps
158+
- [Backend](/ai-chat/backend) — Backend approaches in detail
159+
- [Frontend](/ai-chat/frontend) — Transport setup, sessions, client data
160+
- [Features](/ai-chat/features) — Per-run data, deferred work, streaming, subtasks
161+
- [API Reference](/ai-chat/reference) — Complete reference tables

0 commit comments

Comments
 (0)