feat(agents): persistent Lakebase threads + chat-history UI by MarioCadenas · Pull Request #576 · databricks/appkit

MarioCadenas · 2026-09-04T15:34:18Z

What

Persistent agent threads for AppKit, end to end — a durable ThreadStore plus the client hooks and components for a chat-history sidebar.

Two layers, designed to be splittable (see "Splitting" below):

Backend persistence — LakebaseThreadStore (Postgres over createLakebasePool()), self-bootstrapping schema, per-user isolation.
History UI — summary/rename endpoints + useAgentThreads / useAgentThread hooks + <ThreadList> / <AgentThread> components (@databricks/appkit-ui/react/beta).

How it works

Store (LakebaseThreadStore) — raw parameterized pg SQL (not the DatabasePlugin layer). Two user_id-scoped tables (agent_threads + agent_messages, FK ON DELETE CASCADE), self-bootstrapped with CREATE TABLE IF NOT EXISTS on init(). Every query filters WHERE user_id = $ — the isolation boundary. tool_calls stored as jsonb, preserving thoughtSignature verbatim. Owns its pool by default; leaves an injected pool alone.

History backend — GET /threads returns cheap summaries ({id, title, updatedAt, messageCount}, no message bodies — a LATERAL join derives the title from the first user message via COALESCE(title, …)); GET /threads/:id returns the full thread; PATCH /threads/:id renames (nullable title column, derived default, rename wins, doesn't bump updated_at). ThreadStore gains optional listSummaries/rename (custom stores keep compiling; the plugin falls back to list()).

Client (@databricks/appkit-ui/react/beta) — sibling hooks: useAgentThreads (list + optimistic delete/rename) and useAgentThread (one conversation's transcript; loads history, streams via useAgentChat using a new additive initialThreadId seam to resume). Drop-in components <ThreadList> + <AgentThread>; the page holds the active thread id and wires them (onTurnComplete → refresh the list). No shared provider.

Verification

67 unit tests (store, both hooks, useAgentChat seam, <ThreadList>), all green. pnpm -r typecheck clean vs baseline; check:fix, build, docs:build all pass.
Live-tested against a real Lakebase (dogfood staging, throwaway branch): schema bootstrap incl. ALTER TABLE ADD COLUMN title on a pre-existing table, persistence across a full process restart, per-user isolation (cross-user read/rename → 404), thoughtSignature jsonb round-trip, summary projection, rename (+ empty-title 400), delete/cascade, and the /agent-history route serving.

NOT verified

Deployed service-principal path — live tests ran as my own identity on a throwaway branch, not as the app SP owning the tables in a deployed app. That deploy dogfood is the remaining human step.
UI visual render — the route serves and every endpoint it uses is verified, but I could not eyeball the rendered <ThreadList>/<AgentThread> (no browser in-loop).

Splitting

The 5 UI commits sit cleanly on top of the backend commits, so this can be split into a backend PR + a UI PR if preferred. Kept together here so the whole feature is reviewable at once.

Design

Shaped via a full design review — sibling (not subsumed) hooks per CopilotKit precedent, "thread" naming (ecosystem-universal), list=summaries/detail=full, beta stability tier, derived+renamable titles.

This is a draft — not for merge until the deployed-SP dogfood and a UI eyeball are done.

Persistent ThreadStore backed by Lakebase (Postgres) over raw parameterized pg SQL from createLakebasePool(). Two user_id-scoped tables (agent_threads, agent_messages, FK ON DELETE CASCADE), self-bootstrapping CREATE TABLE IF NOT EXISTS via init(). Every query filters WHERE user_id = the caller's id (the isolation boundary); tool_calls stored as jsonb preserving thoughtSignature verbatim; Dates revived on read. Owns its pool by default, leaves an injected pool alone on close(). Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Add optional init?()/close?() to the ThreadStore interface (in-memory store untouched). The agents plugin awaits threadStore.init?.() first in setup() for fail-fast connectivity and threadStore.close?.() in shutdown() to release an owned pool. Field typed as ThreadStore so the optional hooks resolve. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

…store Declare an optional postgres resource on the agents manifest (same field/env shape as the database plugin, CAN_CONNECT_AND_CREATE) so a deploy can bind Lakebase for persistent threads; unbound apps are unaffected and fall back to in-memory. Export LakebaseThreadStore (and its options type) via the agents barrel and the beta surface, beside the other agents exports. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Add a Thread persistence section to the agents plugin docs (in-memory default vs LakebaseThreadStore, self-bootstrap, per-user isolation, deploy binding, custom-store contract). Wire an opt-in dev-playground example that uses LakebaseThreadStore when LAKEBASE_ENDPOINT is set (same signal the lakebase plugin uses) and falls back to in-memory otherwise. Regenerated API docs and template appkit.plugins.json. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Left-column history panel wired to the existing thread endpoints: GET /api/agents/threads (list), GET /api/agents/threads/:id (load + resume), DELETE /api/agents/threads/:id. New conversation button resets to a fresh thread; the list refreshes after each send and highlights the active thread. Durable when the agent uses LakebaseThreadStore. Demonstrates the persistence from #576 end-to-end in the UI. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Adds the backend for a history sidebar: ThreadSummary type (shared), optional ThreadStore.listSummaries + rename (both impls; optional so custom stores keep compiling), a nullable title column on agent_threads (derived default via COALESCE(title, first user message), renamable), GET /threads now returns summaries (fallback derives from list() for stores lacking listSummaries), and a new PATCH /threads/:id rename route. Rename does not bump updated_at. Extends the InMemory + Lakebase test suites. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Additive, non-breaking option to resume an existing thread: seeds threadId so the first send() continues that thread instead of creating a new one; changing it re-seeds (switch conversations) without clobbering a server-assigned id. This is the internal seam useAgentThread uses to resume persisted threads. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Sibling hooks for agent chat history, exported from @databricks/appkit-ui/react/beta: - useAgentThreads: lists thread summaries (GET {basePath}/threads), with optimistic deleteThread/renameThread; plain-fetch, revives Dates. Owns the list, not the active conversation. - useAgentThread(threadId?): owns one conversation's transcript — loads history (GET /threads/:id), streams turns via useAgentChat (using its initialThreadId seam to resume), commits each completed turn, surfaces the server-assigned id on a new thread, optional URL persistence (default off). Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

…beta) Drop-in components over the two hooks, exported from @databricks/appkit-ui/react/beta: - ThreadList: history sidebar (owns useAgentThreads); controlled selection via activeThreadId/onSelect; per-row dropdown with inline Rename and alert-dialog-confirmed Delete; built on item/scroll-area/dropdown-menu. - AgentThread: one conversation's transcript + composer (owns useAgentThread); resolves the default agent from the plugin client config; onThreadCreated / onTurnComplete callbacks let the page refresh the list (no shared provider). Plain-text messages for v1. Includes a light ThreadList render test. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

Adds an Agent History playground route composing <ThreadList> + <AgentThread> (page holds the active thread id; onTurnComplete refreshes the list), reverting the earlier hand-rolled sidebar from the kitchen-sink /agent route. Documents the hooks + components in the agents plugin doc, and exports ThreadSummary from @databricks/appkit/beta so server-side custom ThreadStore implementers get the type. Regenerated API docs + route tree. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

github-actions · 2026-09-08T14:23:23Z

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

`@databricks/appkit`

npm tarball (packed): 1.1 MB (+29 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	1.1 MB (+33 KB)	408 KB (+10 KB)
Type declarations	417 KB (+11 KB)	150 KB (+4.2 KB)
Source maps	2.3 MB (+63 KB)	764 KB (+19 KB)
Other	11 KB	3.7 KB
Total	3.8 MB (+107 KB)	1.3 MB (+34 KB)

Per-entry composition (own code — deps external (as shipped))

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`.`	95 KB (+119 B)	2.5 KB	98 KB (+119 B)	external	313 KB (+321 B)
`./beta`	86 KB (+4.8 KB)	457 B	86 KB (+4.8 KB)	external	259 KB (+17 KB)
`./testing`	17 KB (+32 B)	0 B	17 KB (+32 B)	external	52 KB (+112 B)
`./tsdown`	520 B	0 B	520 B	external	813 B
`./type-generator`	23 KB	0 B	23 KB	external	65 KB

Chunks:

Entry	Chunk	Load	Size (gz)
`.`	`index.js`	initial	91 KB
`.`	`utils.js`	initial	4.0 KB
`.`	`remote-tunnel-manager.js`	lazy	2.5 KB
`./beta`	`beta.js`	initial	69 KB
`./beta`	`stream-manager.js`	initial	5.8 KB
`./beta`	`wide-event-emitter.js`	initial	3.2 KB
`./beta`	`databricks.js`	initial	3.2 KB
`./beta`	`configuration.js`	initial	2.1 KB
`./beta`	`service-context.js`	initial	1.3 KB
`./beta`	`client.js`	initial	434 B
`./beta`	`client-options.js`	initial	220 B
`./beta`	`supervisor-api.js`	lazy	192 B
`./beta`	`databricks.js`	lazy	142 B
`./beta`	`index.js`	lazy	123 B
`./testing`	`index.js`	initial	17 KB
`./tsdown`	`index.js`	initial	520 B
`./type-generator`	`index.js`	initial	23 KB

`@databricks/appkit-ui` ⚠️ over budget

npm tarball (packed): 368 KB (+19 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	413 KB (+19 KB)	139 KB (+6.7 KB)
Type declarations	238 KB (+9.0 KB)	88 KB (+4.2 KB)
Source maps	805 KB (+39 KB)	267 KB (+14 KB)
CSS	16 KB	3.2 KB
Total	1.4 MB (+66 KB)	498 KB (+25 KB)

Per-entry composition (consumer bundle — deps bundled, peerDeps external)

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
`./js`	5.3 KB	49 KB	55 KB	208 KB	14 KB
`./js/beta`	20 B	0 B	20 B	0 B	0 B
`./react`	432 KB (+5 B)	49 KB	481 KB (+5 B)	1.3 MB (+3 B)	177 KB (+67 B)
`./react/beta`	47 KB (+46 KB)	0 B	47 KB (+46 KB) ⚠️	122 KB (+122 KB)	20 KB (+18 KB)

Chunks:

Entry	Chunk	Load	Size (gz)
`./js`	`index.js`	initial	5.2 KB
`./js`	`chunk`	initial	120 B
`./js`	`apache-arrow`	lazy	49 KB
`./js/beta`	`beta.js`	initial	20 B
`./react`	`index.js`	initial	430 KB
`./react`	`tslib`	initial	2.1 KB
`./react`	`apache-arrow`	lazy	49 KB
`./react/beta`	`beta.js`	initial	47 KB

⚠️ Over budget: a package's shipped tarball, or a browser entry's consumer bundle (deps included), grew by more than 5% (and >10 KB). This check will fail — reduce the size, or acknowledge the increase by updating bundle-size-baseline.json.

Re-selecting a thread no longer refetches GET /threads/:id. useAgentThread keeps a per-hook Map<threadId, messages> cache: served on switch when present, written through as turns are sent/streamed so a cached entry stays current (no explicit invalidation). The playground history route also drops the per-select remount so the cache survives — switching threads is now instant and quiet. Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

github-actions · 2026-09-08T14:39:44Z

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 34238867126 -R databricks/appkit -n appkit-template-0.72.0-pr.6232659-feat-agent-persistent-threads-576 -D appkit-pr-576 \
  && unzip -o "appkit-pr-576/appkit-template-0.72.0-pr.6232659-feat-agent-persistent-threads-576.zip" -d "appkit-pr-576" \
  && databricks apps init --template "appkit-pr-576"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

MarioCadenas added 11 commits September 4, 2026 17:35

chore(playground): gitignore local scratch/ dir

0d7e8b3

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>

MarioCadenas force-pushed the feat/agent-persistent-threads branch from b01bf00 to 5c2e834 Compare September 8, 2026 14:20

MarioCadenas changed the title ~~feat(agents): persistent Lakebase thread store~~ feat(agents): persistent Lakebase threads + chat-history UI Sep 8, 2026

Navigation Menu

Uh oh!

feat(agents): persistent Lakebase threads + chat-history UI - #576

feat(agents): persistent Lakebase threads + chat-history UI#576
MarioCadenas wants to merge 12 commits into
mainfrom
feat/agent-persistent-threads

MarioCadenas commented Sep 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 8, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

MarioCadenas commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How it works

Verification

NOT verified

Splitting

Design

Uh oh!

github-actions Bot commented Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Bundle size report

@databricks/appkit

@databricks/appkit-ui ⚠️ over budget

Uh oh!

github-actions Bot commented Sep 8, 2026

🤖 AppKit PR bot

🔬 Run evals

📦 Try this PR's app template

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

MarioCadenas commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 8, 2026 •

edited

Loading

`@databricks/appkit`

`@databricks/appkit-ui` ⚠️ over budget