feat(chat)!: serve the AG-UI chat instead of Chainlit - #37
Merged
Conversation
Closes #31. The hosted chat was the runtime's Chainlit host, which showed a conversation and nothing else. Everything this repository exists to demonstrate — where a tool's arguments came from, what the toolsets exchanged without the model reading it, a tool's own view — happened where nobody could see it. It is now `mcp_agent_api`: the agent over HTTP as AG-UI events, and the web client that renders them, both inside the runtime wheel as of 0.8.4. So the image installs one package and runs uvicorn — no node stage, no vendored frontend, no `install-elements` step after a runtime bump. Chainlit is gone from this repo entirely, the root pin included. **The model is now the deployment's.** Chainlit was bring-your-own-model because it had a settings dialog to type into; this agent is built once at startup, so whoever opens the page spends the deployment's key. Rather than quietly hold one, neither target deploys the chat without one: - Kubernetes: the `MCP_PROVIDER_API_KEY` secret gates the job, and the chart templates a Secret from it (or takes one you manage). `--set-file`, because helm reads a comma in `--set` as a separator and would truncate a key. - AWS: the `MCP_AWS_CHAT_MODEL` variable is the on switch, and the key is a Parameter Store SecureString read at task start — passing it as context would put it in the CloudFormation template. Per-toolset credentials survive the move: the new `GET /connections` route reports which headers the toolsets declared, the page offers a field per header, and a visitor's value beats the deployment's. Two things found while wiring it up rather than by reading: A DNS record was written for a chat that is not deployed — a hostname that resolves to the load balancer with no rule behind it lands on the index, so the chat host answers with the directory. One predicate now answers all three questions that have to agree: the certificate's second name, the listener rule and the record. The AWS health check was on `/`, which the old chat had no better answer for. The page is served as soon as the process is up and stays unable to answer until it has connected to every toolset, so the check is now `/health/readiness` and a task joins the load balancer when it can actually answer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing writes .chainlit/, chainlit.md or public/elements any more: the chat image runs uvicorn and the web client is inside the wheel. An ignore for a file nothing produces is a reader's clue to code that is gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two targets gated on different halves of the same decision: Kubernetes on the key, AWS on the model. Each then had a sharp edge where the other half was missing — a key with no model failed inside `helm upgrade`, and a model with no key started a task that could not run and rolled back. Neither said what to set, and the asymmetry had no reason behind it beyond how each was written. The model is now the switch on both, because it is the one input both can see: a job-level `if` cannot read secrets, and synthesis is credential-free by rule, so the stack cannot know whether a Parameter Store entry exists. What both *can* do is check the key early and say so — Kubernetes reads the secret in a first step, AWS reads the parameter's version without decrypting it, and each error names the exact command to run or the variable to clear. That leaves one decision with one switch. A repository made from this template still deploys no chat until someone names a model, which was the point. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reviewing the branch rather than reading it, starting from what a container actually does at startup. uvicorn opens its socket *after* the lifespan, and the lifespan is where the agent connects to every toolset — so during startup this container refuses connections rather than answering 503, which is what two comments here claimed and neither target was set up for. - Kubernetes had a liveness probe and no startup probe, so its own failures would restart a container that was still connecting. A slow toolset would have produced a crash loop that reads as a broken image. There is a startup probe now, with the liveness and readiness checks behind it. - ECS took the library's 60s grace period, which is right for a toolset that serves the moment it starts and thin for one that connects to every toolset first. The chat gets three minutes; nothing else changes. Then four smaller ones, each found by trying it: - A `ui.examples` entry spanning lines broke out of the block scalar and the whole Deployment failed to parse. Joined and quoted instead. - `provider.existingSecret` with a blank `existingSecretKey` rendered `key:` with nothing after it. It is `required` now. - `containerPort` followed `service.port`, so changing the Service's port pointed the probes and the Service at a port the image does not serve. The image fixes 8080; so does the container port, and the Service targets it by name. - Both new error messages substituted a repository variable into the script body. They go through the environment now. The AWS one also told you to create a parameter whatever went wrong, including an expired token; it tells ParameterNotFound from the rest. And two doc leaks of my own: a Kubernetes-only repo learned about `MCP_AWS_CHAT_MODEL` and `infra/cdk`, because the sentence naming both targets was outside any marker. Marked. (The comparison table and a few lines of prose elsewhere leak the same way and predate this branch.) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A typo in MCP_AGENT_UI_TITLE fails silently and identically on both targets: the deployment sets a variable nothing reads, the client falls back to its own default, and the page looks merely unconfigured. Nothing here would have noticed — a chart render and a stack synthesis both know what they set and neither knows what the runtime is looking for. The test reads the names out of `mcp_agent_api.ui` and asserts each target names exactly those, so a variable that nothing reads fails as loudly as a missing one. Text rather than imports, because `infra/cdk/config.py` needs the `infra` group and the chart is YAML — one test covers both targets in the base job, and follows whichever target survived a bootstrap. Verified by breaking it both ways: a renamed variable in the chart and one in the CDK config each fail it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…more - Kubernetes: the reconcile job uninstalls mcp-chat when MCP_CHAT_MODEL is empty. The chat job was merely skipped, so a cleared variable left the release running and spending the key, while the AWS stack reconciled. - AWS: the preflight fails on a model with no MCP_AWS_INGRESS_HOST, which previously deployed no chat and said nothing, and now hands the stack the parameter path it checked (chatApiKeyParameter), so the two agree. - Chart: a model with no apiKey or existingSecret fails at install with a message, instead of a pod waiting forever on a Secret never written. - .dockerignore: both Dockerfiles copy the whole tree into a builder stage; .venv, node_modules, cdk.out, .env and any key file no longer go with it. - .example.env: drop the stale mcp-agent-web name and put the hosted-chat note under the two variables it describes. - chat_host docstring states the rule rather than a bug main never had. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #31.
The hosted chat was the runtime's Chainlit host, which showed a conversation and nothing else. Everything this repository exists to demonstrate — where a tool's arguments came from, what the toolsets exchanged without the model reading it, a tool's own view — happened where nobody could see it.
It is now
mcp_agent_api: the agent over HTTP as AG-UI events, and the web client that renders them, both inside the runtime wheel as of 0.8.4. The image installs one package and runs uvicorn — no node stage, no vendored frontend, noinstall-elementsafter a runtime bump. Chainlit is gone from this repo entirely, the root pin included.The model is now the deployment's
Chainlit was bring-your-own-model because it had a settings dialog to type into. This agent is built once at startup, so whoever opens the page spends the deployment's key. Rather than quietly hold one, neither target deploys the chat without one — a repository made from this template has no chat until someone puts a key there deliberately.
MCP_PROVIDER_API_KEYsecretprovider.existingSecretMCP_AWS_CHAT_MODELvariable/mcp-toolsets/<instance>/chat/provider-api-keyThe key is passed with
--set-fileon the Kubernetes side, because helm reads a comma in a--setvalue as a separator and would truncate a key into something that looks merely rejected. On AWS it is never CDK context, which would put it in the CloudFormation template.Per-toolset credentials survive the move. The runtime's new
GET /connectionsreports which headers the toolsets declared, the page offers a field per header, and a visitor's own value beats the deployment's.Two things found by wiring it up
/. The page is served as soon as the process is up and cannot answer until it has connected to every toolset, so a task joined the load balancer while every question it got would 503. It is/health/readinessnow.Checks
./scripts/lint,./scripts/test(35), the CDK tests (43, four of them new),helm linton the chat chart — now part of CI — and a synth both ways confirming the chat appears with a model and no chat resources or record appear without one, with an emptymissingin the manifest either way.🤖 Generated with Claude Code