diff --git a/context/agents/integration-v2/ai-observability.md b/context/agents/integration-v2/ai-observability.md new file mode 100644 index 00000000..03605f22 --- /dev/null +++ b/context/agents/integration-v2/ai-observability.md @@ -0,0 +1,39 @@ +--- +type: ai-observability +flow: integration-v2 +label: Instrument LLM calls +model_pi: openai/gpt-5.6-terra +effort_pi: medium +model_sdk: claude-sonnet-4-6 +effort_sdk: high +skills: [llm-analytics-setup, posthog-best-practices] +allowedTools: [Read, Write, Edit, Glob, Grep] +disallowedTools: [Bash, enqueue_task] +dependsOn: [install, init] +--- + +## Goal + +Route this app's LLM calls through PostHog so each generation reports its model, +cost, latency, and token counts. The provider or framework in use decides the shape: +a wrapped client, a callback handler, a decorator. Follow the reference for the +provider this project actually imports, and reach the PostHog client the way the init +step already established — never construct a second one. + +Instrument at the client, not at the call sites: one wrapped client that every call +site already reaches beats editing each call. If a call site must change, keep the +change to the smallest edit that routes it through the wrapped client. + +Where the app has a user identity available at the call, pass it as the distinct id +so generations attribute to a person. Where it does not, leave it out rather than +inventing one — an anonymous generation is still useful, a wrong identity is not. +Never send prompt or completion content the app treats as sensitive. + +## How you know you succeeded + +Every LLM entry point in the app goes through the instrumented client, the project +still builds the way it did before you started, and no second PostHog client exists. +Your handoff names the files you changed, the provider or framework each one uses, +whether a distinct id was available, and what a reviewer should read to confirm the +call path. If the project imports an LLM SDK you found no reference for, say so and +leave it uninstrumented rather than guessing at an API. diff --git a/context/agents/integration-v2/capture.md b/context/agents/integration-v2/capture.md deleted file mode 100644 index 97f60d88..00000000 --- a/context/agents/integration-v2/capture.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -type: capture -flow: integration-v2 -label: Capture events -model_pi: openai/gpt-5.6-terra -effort_pi: medium -model_sdk: claude-sonnet-4-6 -effort_sdk: high -skills: [integration-v2-capture, posthog-best-practices] -allowedTools: [Read, Write, Edit, Glob, Grep] -disallowedTools: [enqueue_task] -dependsOn: [install, init, identify] ---- - -## Goal - -Decide which events are worth capturing in this app, then instrument them in the -same pass — read each file once, choose the events, and add the capture calls -while the file is already open. Instrumenting the events is the job; everything -below is a rule you apply while you do it, not work of its own. - -The identify step's handoff already says how this app attributes events. Read it -once, up front, and follow it: - -1. Where identity is established for you — a client library that remembers it, or a - framework middleware that binds it per request — a plain capture is already - attributed. Write the plain call and move on. Do not add a context of your own, - and do not go looking through the call tree to prove it. -2. Where the handoff says it is not — no middleware, or a call path that runs - outside the request — tag that call with the distinct id and the session id. - -Where the project has no identity to begin with — the identify step found nothing to -wire, because the app has no accounts or login — personless events are the right -answer. Capture plainly, with no id and no placeholder, and say so in your handoff. - -The placeholder is for the third case only: identity exists in this project, but no -stable id reaches the call site. Do not guess at the id and do not invent a fallback — -pass `DISTINCT_ID` there so the gap is visible in the code, and say in your handoff -that identity exists but could not be reached, so the report calls it out as an issue -to follow up rather than claiming events that belong to nobody. - -Never put PII in an event. When new information about the user surfaces, tag the -user the way the identify docs describe, not the event. - -## How you know you succeeded - -The meaningful user actions across the app have capture calls that fire on the -real action, not on page load, each one attributable to the user who took it, and -`.posthog-wizard-cache/.posthog-events.json` lists the events you instrumented. diff --git a/context/agents/integration-v2/dashboard.md b/context/agents/integration-v2/dashboard.md deleted file mode 100644 index b0dfecef..00000000 --- a/context/agents/integration-v2/dashboard.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -type: dashboard -flow: integration-v2 -label: Create a starter dashboard -model_pi: openai/gpt-5.6-luna -effort_pi: low -model_sdk: claude-sonnet-4-6 -effort_sdk: high -skills: [integration-v2-dashboard, integration-v2-insight, integration-v2-mcp] -allowedTools: [Read, Glob, Grep] -disallowedTools: [Write, Edit, Bash, enqueue_task] -dependsOn: [capture] ---- - -## Goal - -Create a starter PostHog dashboard with a few insights built on the events this -integration instruments, using the PostHog MCP. - -## How you know you succeeded - -A dashboard exists with a handful of insights on the captured events, and you hand -off its URL for the report to link. diff --git a/context/agents/integration-v2/enable-features.md b/context/agents/integration-v2/enable-features.md new file mode 100644 index 00000000..20e23f65 --- /dev/null +++ b/context/agents/integration-v2/enable-features.md @@ -0,0 +1,42 @@ +--- +type: enable-features +flow: integration-v2 +label: Enable the PostHog products +model_pi: openai/gpt-5.6-luna +effort_pi: low +model_sdk: claude-haiku-4-5-20251001 +skills: [integration-v2-enable-features, integration-v2-mcp] +allowedTools: [Read, Glob, Grep] +disallowedTools: [Write, Edit, Bash, enqueue_task] +dependsOn: [product-signals] +--- + +## Goal + +Turn on the PostHog products that self-driving reads from, in the project. The SDK is +installed and initialized by now, so this is the switch that decides whether what it +sends is collected at all. `init` queued you behind the repo groom, so its findings are +already in your context. + +Enable the products, then say what the groom's findings change about them. You change +nothing in the repo — a product that needs code to work is a follow-up you name, not an +edit you make. + +## Status + +Complete with **done** whenever you established each product's state and recorded it — +including when PostHog refused the change. A missing tool, a missing scope, or a +permission wall is an outcome to report, not a failure: the integration in the repo is +untouched and every later step can still run, so failing here would abort a run that +actually succeeded. Reserve **failed** for being unable to determine any product's state +at all. + +## How you know you succeeded + +Every product has a recorded result — enabled, already enabled, or refused with the +reason. Your handoff names each one, carries forward the `product-signals` block +verbatim so the steps after you see it by the shortest path, and lists every follow-up: +a product needing project admin or a broader token scope, a product inert on this +platform until the SDK is configured for it, a Support inbox with no channel connected +yet. If no product signals were available, say that plainly rather than implying you had +them. diff --git a/context/agents/integration-v2/init.md b/context/agents/integration-v2/init.md index cfbba467..a9444e6c 100644 --- a/context/agents/integration-v2/init.md +++ b/context/agents/integration-v2/init.md @@ -8,15 +8,34 @@ model_sdk: claude-sonnet-4-6 effort_sdk: medium skills: [integration-v2-init, posthog-best-practices] allowedTools: [Read, Write, Edit, Glob, Grep] -disallowedTools: [enqueue_task] +disallowedTools: [] dependsOn: [] --- ## Goal -Initialize PostHog: create the framework's init point so the SDK is configured -once and available across the app, set the PostHog environment variables through -the wizard tools, and document those keys in `.env.example` for other developers. +Before your own work, queue exactly two tasks, once each, then move on and never wait on +either: + +1. `product-signals`, with no dependencies, so it grooms the repo in parallel while you + edit. Pass what you already know through its `inputs`: the framework, the kind of app + (client, SSR/fullstack, or server), the init file you are about to create or edit, and + whether the app ships a Content-Security-Policy. +2. `enable-features`, depending on the id `product-signals` just returned — it turns the + products on in PostHog, and the groom's findings are what tell it what it is turning + on. Nothing else depends on it. + +Those two types and nothing else. The rest of the run is already queued — the planner +seeded install, identify, error tracking, review, and the report before you started, so +queueing any of them again duplicates work that is already running. Work you think is +missing goes in your handoff, not on the queue. + +You read neither result; the steps after you do. + +Then initialize PostHog: create the framework's init point so the SDK is configured +once and available across the app, turn on the products that are init options rather +than steps of their own, set the PostHog environment variables through the wizard +tools, and document those keys in `.env.example` for other developers. Try to follow these principles, in order: codebase convention, don't make any unnecessary changes, keep the change as lean as possible for an easy review, and @@ -25,7 +44,9 @@ part of the shape: reach it the way the docs and the example project reach it. ## How you know you succeeded -The init file exists and the PostHog env keys are present. Keys live in the env +A `product-signals` task and an `enable-features` task that depends on it are both on +the queue. The init file exists and the PostHog env +keys are present. Keys live in the env file, never hardcoded in source, and `.env.example` lists the key names (with placeholder values) so the next developer knows what to set. Your handoff names the files you changed, how the client is constructed in them, and how a call site diff --git a/context/agents/integration-v2/integrate-posthog.md b/context/agents/integration-v2/integrate-posthog.md index 18ba3c1e..2a3a6129 100644 --- a/context/agents/integration-v2/integrate-posthog.md +++ b/context/agents/integration-v2/integrate-posthog.md @@ -16,23 +16,37 @@ dependsOn: [] Plan a PostHog integration and seed the task queue with this graph: -- `install` and `init`, independent of each other. +- `install` and `init`, independent of each other. `init` also configures the products + that are init-time options rather than steps of their own — autocapture, pageviews, + session replay — and owns two tasks you do not queue: the repo groom + (`product-signals`) and the project-side switch-on (`enable-features`) that reads it. + Leave both to `init`; queueing either here would run it before the groom exists. - `identify` and `error-tracking`, each after `install` and `init` and independent of one another, so they run in parallel. `error-tracking` makes the errors the app does not catch reach PostHog, by whatever means the SDK offers for that — it needs the SDK installed and initialized, not the events. -- `capture`, after `identify` — it decides the events and instruments them, and it - reads how identity is already established before it instruments anything. -- `review`, after `install`, `init`, `identify`, `capture`, and `error-tracking` — - it installs the dependencies, verifies the project builds/typechecks/lints, and - reviews every change the run made, fixing what fails. There is no separate build - step: verifying and reviewing are one pass over the same changeset. -- `dashboard`, after `capture`, parallel to `review` — it builds insights from the - instrumented events, which `capture` has already defined; it needs no code review. -- `report`, after `dashboard` **and** `review` — it writes the setup report last, so - it describes the integration as reviewed rather than as first written. +- `ai-observability`, after `install` and `init`, **only if this project actually + calls an LLM**. Read the manifests before you decide: an LLM SDK or framework in the + dependencies (`openai`, `@anthropic-ai/sdk`, `ai`, `@ai-sdk/*`, `langchain`, + `cohere`, `mistralai`, `google-genai`, or their Python equivalents in + `requirements.txt` / `pyproject.toml`). Found one, queue it; found none, leave it + out — omitting it is the right plan for an app with no LLM calls, not a gap. +- `review`, after `install`, `init`, `identify`, `error-tracking`, and + `ai-observability` if you queued it — it installs the dependencies, verifies the + project builds/typechecks/lints, and reviews every change the run made, fixing what + fails. There is no separate build step: verifying and reviewing are one pass over + the same changeset. It does not wait on the project-side switch-on, which touches no + code. +- `report`, after `review` — it writes the handoff last, so it describes an integration + that has already been reviewed. It reads every step's handoff out of the queue log, + including the tasks `init` queued, so it needs no edge to them. + +This run does not instrument events and does not build insights or dashboards. That is +deliberate: it leaves the user's own agent the tools and the suggestions to do that, +and it is why the run is short. Do not queue work for it. ## How you know you succeeded -Every task in the graph is queued with that dependency shape, the report last, -and the first task runnable. Keep labels short — the action in a few words. +Every task in the graph is queued with that dependency shape, the report last, and the +first task runnable. `ai-observability` is present exactly when the manifests justify +it. Keep labels short — the action in a few words. diff --git a/context/agents/integration-v2/product-signals.md b/context/agents/integration-v2/product-signals.md new file mode 100644 index 00000000..aa1e7305 --- /dev/null +++ b/context/agents/integration-v2/product-signals.md @@ -0,0 +1,65 @@ +--- +type: product-signals +flow: integration-v2 +label: Look for product signals +model_pi: openai/gpt-5.6-luna +effort_pi: medium +model_sdk: claude-haiku-4-5-20251001 +effort_sdk: medium +skills: [] +allowedTools: [Read, Glob, Grep] +disallowedTools: [Write, Edit, Bash, enqueue_task] +dependsOn: [] +--- + +## Goal + +Read this repo for evidence about which PostHog products would actually earn their +place here, and hand that evidence forward. You change nothing and enable nothing — +the steps after you decide what to do with what you find. + +Evidence, not opinions. Every line you report names the file it came from, and a +claim you could not read out of a file is an assumption, not a finding. Look for: + +- what the app is for — routes, pages, entry points, and whether any of them already + call an analytics library +- an LLM client or framework, and the call sites that use it +- a structured logger, and where it writes +- a payments or billing SDK, and the webhook or checkout path it serves +- an existing error reporter already shipping exceptions somewhere +- tenancy: an organizations/teams/workspaces table, or a foreign key on users that + implies one +- a database or third-party service the warehouse could ingest +- public marketing routes, as distinct from the authenticated app + +Read broadly and cheaply: the manifests first, then the schema, then the entry +points, then the specific files those point at. Do not read the whole tree, and do +not open a dependency's own source. + +## How you know you succeeded + +Your handoff's `forNextAgent` is one line per product, in this exact shape, with +every product present — a reader has to be able to tell "looked, found nothing" from +"did not look": + +``` +: yes|no|partial — () +``` + +Use `partial` when PostHog would overlap something the project already runs; that is +the case the later steps handle differently from a plain `yes`. Cover exactly these +products, in this order: `product-analytics`, `session-replay`, `error-tracking`, +`logs`, `ai-observability`, `revenue`, `groups`, `data-sources`, `web-analytics`. + +For example: + +``` +product-analytics: yes — 14 routes under app/, no analytics calls anywhere (app/, package.json) +error-tracking: partial — @sentry/nextjs 8.x already reporting (package.json, sentry.client.config.ts) +groups: no — single-tenant, no org or team table (prisma/schema.prisma) +``` + +`did` says where you looked and what you read. `evidence` names the greps and reads +you actually ran. `assumptions` carries anything inferred from a name rather than +read — an `organizations` table taken to mean tenancy. Leave `filesTouched` out; you +touch nothing. diff --git a/context/agents/integration-v2/report.md b/context/agents/integration-v2/report.md index 66c3b9a3..c00ad4cc 100644 --- a/context/agents/integration-v2/report.md +++ b/context/agents/integration-v2/report.md @@ -9,35 +9,43 @@ effort_sdk: high skills: [integration-v2-report, integration-v2-notebook, integration-v2-mcp] allowedTools: [Read, Write, Glob, Grep] disallowedTools: [enqueue_task] -dependsOn: [dashboard, review] +dependsOn: [review] --- ## Goal -Write the setup report summarizing what this integration did, drawing only on what -the run itself recorded: the queue log and event plan in `.posthog-wizard-cache/` -(`queue.json` and `.posthog-events.json`), and the handoff each step left behind. -Then mirror it into a shareable PostHog notebook. +Write the handoff for this project — half a record of what the run did, half a brief +for the agent that picks up after it. Draw only on what the run itself recorded: the +queue log at `.posthog-wizard-cache/queue.json`, which holds every step's handoff +inline, the `product-signals` groom among them. Then mirror it into a shareable +PostHog notebook. -Separate what the run verified from what it did not. A passing build proves the -code compiles, not that events flow — never write that an event was captured -unless the run observed it arrive. Where a step failed or was skipped, say so -plainly instead of rounding it up to success. Anything only the user can confirm -belongs in a checklist to work through before merging, each item naming the file -and line to look at. +This run deliberately instrumented no events and built no insights or dashboards, so +the report is where that work gets handed over: the events worth capturing, the +analyses they would unlock, the data sources worth connecting, and prompts the user +can paste straight into their own agent. Ground every suggestion in the +`product-signals` findings and the files this run actually touched. If that groom is +missing from the queue log, say so in those sections rather than filling them with +guesses. + +Separate what the run verified from what it did not. A passing build proves the code +compiles, not that events flow — never write that an event was captured unless the run +observed it arrive, and this run captured none. Where a step failed or was skipped, say +so plainly instead of rounding it up to success. Anything only the user can confirm +belongs in a checklist to work through before merging, each item naming the file and +line to look at. Where a handoff reports something a step could not resolve — attribution it could not -establish, a question the step before it left open — that is not a caveat to bury in -prose. Raise it as its own issue to follow up, saying what is unresolved and what it -costs if left alone. A `DISTINCT_ID` placeholder left at a call site means no stable id -was available: name every file and line carrying one, so the user knows what to replace -before those events mean anything. +establish, a product it could not enable for lack of permissions, a question the step +before it left open — that is not a caveat to bury in prose. Raise it as its own issue +to follow up, saying what is unresolved and what it costs if left alone. ## How you know you succeeded -`posthog-setup-report.md` exists at the project root: what was installed and -initialized, the events captured, whether identify was wired or skipped, error -tracking added, the dashboard link, any build conflict in full, and the next -steps for the user. Every claim in it traces to a handoff, and what the run could -not confirm reads as unconfirmed. The report is also mirrored into a PostHog -notebook whose URL is emitted with the `[NOTEBOOK_URL]` marker. +`posthog-setup-report.md` exists at the project root, in the section order the report +skill lays out: what is set up, what was left for their agent, suggested events, the +analyses those events unlock, how to drive PostHog from an agent, copy-paste prompts, +suggested data sources, the before-you-merge checklist, and what the run did not do and +why. Every claim traces to a handoff, every suggestion traces to a signal or a file, +and what the run could not confirm reads as unconfirmed. The report is also mirrored +into a PostHog notebook whose URL is emitted with the `[NOTEBOOK_URL]` marker. diff --git a/context/agents/integration-v2/review.md b/context/agents/integration-v2/review.md index 53370581..90d8adc6 100644 --- a/context/agents/integration-v2/review.md +++ b/context/agents/integration-v2/review.md @@ -9,7 +9,7 @@ effort_sdk: high skills: [posthog-best-practices, integration-v2-build] allowedTools: [Read, Edit, Glob, Grep, Bash] disallowedTools: [enqueue_task] -dependsOn: [install, init, identify, error-tracking, capture] +dependsOn: [install, init, identify, error-tracking] --- ## Goal @@ -57,22 +57,22 @@ order: codebase. Codebase idiom beats the example when they conflict; the PostHog correctness rules you were given beat both. -You never change what is captured — event names, properties, and where events fire -are a contract the dashboard and report are built on. Refactor how, and kill bad -changes; never alter the what. - -Events reaching PostHog in production is the point of this code, so a capture wired -to a place it will never fire, an uninitialized SDK, or a call the runtime silently -drops is squarely yours to catch and fix — it is a code defect. What you cannot do is -confirm delivery: you cannot exercise the app, and you never treat "I could not -verify events arrive" as a finding. +This run instruments no events, so there is no event contract to police. What has to +work is the path that would carry them: an SDK that initializes, one client every call +site can reach, identity established where the app has one, and unhandled errors +leaving the process. An uninitialized SDK, a second client constructed alongside the +first, or a call the runtime silently drops is squarely yours to catch and fix — it is +a code defect. What you cannot do is confirm delivery: you cannot exercise the app, +and you never treat "I could not verify events arrive" as a finding. Stay inside verify-and-review. These are not your job: -- The report or the dashboard — other tasks own those; do not write or judge them. -- Product or analytics design — which events are worth tracking, identity strategy, - framework-version tradeoffs. The changeset is the decision; review whether the code - implements it correctly, not whether it was the right decision. +- The report — another task owns it; do not write or judge it. +- The project's PostHog settings — a different task turns the products on through + PostHog and changes no code, so none of it is in your changeset. +- Product or analytics design — which events would be worth tracking, identity + strategy, framework-version tradeoffs. The changeset is the decision; review whether + the code implements it correctly, not whether it was the right decision. - Anything the integration did not touch. Do not widen scope to pre-existing issues. If you notice something real but out of scope, leave it — a one-line mention in your diff --git a/context/shared/enable-products.md b/context/shared/enable-products.md new file mode 100644 index 00000000..e33d8511 --- /dev/null +++ b/context/shared/enable-products.md @@ -0,0 +1,28 @@ +Turn ON the PostHog products that self-driving reads from — **Session Replay**, **Error Tracking**, and **Support** (Conversations). A product left off just sits idle: the sources that consume it have nothing to read. This is a switch in PostHog, not a change in the repo. + +Reach `products-enable` through the PostHog `exec` tool. Discover before you call — catalogs differ by project and by what the run's token is scoped for, so never assume the tool is there: + +``` +search product +info products-enable +``` + +Then enable all three in one call: + +``` +call products-enable { "products": ["session_replay", "error_tracking", "conversations"] } +``` + +It is idempotent and server-owned — you pass no settings, because the server owns each product's enable recipe. The response is `{ "results": { : "enabled" | "already_enabled" } }`. Enabling Support also mints its widget token, leaving the widget itself off until a channel is connected. Record the per-product result. + +**When `products-enable` isn't in the catalog.** That means this run's token was not granted `product_enablement:write`, the purpose-built scope the tool sits behind. Read the current state instead so the report can still tell the truth — `call project-get {"id": "@current"}`, where `session_recording_opt_in` is Session Replay, `autocapture_exceptions_opt_in` is exception capture, and `conversations_enabled` is Support — then record a follow-up saying the products need enabling from PostHog directly. Do **not** try to write those fields with `project-settings-update`: it requires the far broader `project:write`, which this run deliberately does not hold, and a 403 there tells you nothing new. + +Refusals are expected and are not your failure. A tool absent from the catalog, a missing scope, or a user who isn't a project admin all end the same way: record the state you could read, raise a follow-up naming exactly what has to change, and carry on. Never abort, never reach for a raw REST call, and never invent a tool name. + +Then check what the client does with those switches: + +- **Web app** (this repo serves a browser frontend / loads `posthog-js`): the switch only takes effect if the client init doesn't override it. Find the `posthog.init(...)` call and check its options. `disable_session_recording: true` cancels the replay switch; `capture_exceptions: false` cancels the error-tracking one. If neither is set, the switch is enough. +- **Pure backend or mobile app** (no `posthog-js` reads the server config): the switch is inert until the SDK is configured in code. Record a follow-up noting that replay / exception capture for this platform needs SDK changes. +- **Support / Conversations** only produces tickets once an inbound channel (email / inbox / Slack) is connected. You are not connecting a channel here — record it as a next step for the user. + +Record every result and any follow-up. diff --git a/context/skills/integration-v2/build/description.md b/context/skills/integration-v2/build/description.md index c21d8171..a7ac159f 100644 --- a/context/skills/integration-v2/build/description.md +++ b/context/skills/integration-v2/build/description.md @@ -71,6 +71,5 @@ Complete this task with status **done** whenever the integration itself is in place — even if the build or typecheck still fails on pre-existing errors in files you never touched. Note the pre-existing failure in `conflict` and move on. Only use status **failed** when your own integration changes are what break the build -and you cannot resolve them. A `failed` status stops later steps that depend on -this one (the dashboard and report), so do not fail the task over breakage the -integration did not cause. +and you cannot resolve them. A `failed` status stops the report that depends on this +one, so do not fail the task over breakage the integration did not cause. diff --git a/context/skills/integration-v2/capture/config.yaml b/context/skills/integration-v2/capture/config.yaml deleted file mode 100644 index b0c5cb51..00000000 --- a/context/skills/integration-v2/capture/config.yaml +++ /dev/null @@ -1,10 +0,0 @@ -# Orchestrator step-skill: fetched by task agents, never surfaced as a command. -# The framework matrix is borrowed from the canonical integration group. -type: docs-only -template: description.md -description: Instrument the planned events with PostHog capture calls -tags: [orchestrator, capture] -cli: - role: internal -variants_from: integration -packaging: bundle diff --git a/context/skills/integration-v2/capture/description.md b/context/skills/integration-v2/capture/description.md deleted file mode 100644 index dbb68123..00000000 --- a/context/skills/integration-v2/capture/description.md +++ /dev/null @@ -1,50 +0,0 @@ -# Plan and capture events - -Decide which custom events are worth capturing, then instrument them — in one -pass, reading each file once. - -## Choose and record - -From the project's files, find the actions that have business value for event -tracking — especially conversion and churn events. Read them. Track actions, not -pageviews (autocapture covers those). Server-side events matter most where there -is instrumentable server-side code (API routes, server actions): payment/checkout -completion, webhook handlers, and auth endpoints. - -Around ten to fifteen is a rough guide, not a quota. Fewer is right when a small -project has only a few actions that matter — instrument those and stop; never -invent an event to reach a number, since an event nobody performs is noise in -every insight built on it. More than fifteen genuinely valuable events (auth, -payment, and the like) is also fine, but do not put them all in this run — a -first integration PR that touches everything is hard to review. Instrument the -core set now, and record the rest as suggestions for the setup report. - -First scan for capture calls the project already makes, and note how their event -names are formatted. Event names, property names, and feature flag keys are an -analytics contract: reuse the existing names and follow the patterns already in -the project rather than inventing parallel ones, and don't duplicate events that -already exist. - -Write the chosen events to `.posthog-wizard-cache/.posthog-events.json` — a JSON -array of `{ event, description, file }`, one entry per event. That cache directory -is the wizard's, already created for the run; write the plan there, not at the -project root. Write it before you start editing: it drives the event-plan view, -and it is the source the report reads later. - -## Instrument - -For each event call the SDK's capture method on the real user action — the click -or submit handler, the server action — not on render or page load. Use clear -`lower_snake_case` names and useful properties. Edit each file while it is already -open. - -Server-side, use the authenticated user's id as the distinct id. For a genuinely -unauthenticated action, emit a personless event — never fabricate a placeholder -id like `'anonymous'`, which collapses every anonymous user into one person and -corrupts the data. - -Leave `.posthog-wizard-cache/.posthog-events.json` in place for the report. - -## Reference - -{references} diff --git a/context/skills/integration-v2/dashboard/config.yaml b/context/skills/integration-v2/dashboard/config.yaml deleted file mode 100644 index f5cc272e..00000000 --- a/context/skills/integration-v2/dashboard/config.yaml +++ /dev/null @@ -1,11 +0,0 @@ -type: docs-only -template: description.md -description: Create a starter PostHog dashboard from the captured events -tags: [orchestrator, dashboard] -variants: - - id: all - display_name: PostHog dashboard step - tags: [orchestrator, dashboard] - docs_urls: [] -cli: - role: internal diff --git a/context/skills/integration-v2/dashboard/description.md b/context/skills/integration-v2/dashboard/description.md deleted file mode 100644 index a282ac60..00000000 --- a/context/skills/integration-v2/dashboard/description.md +++ /dev/null @@ -1,32 +0,0 @@ -# Create a starter dashboard - -Create a live PostHog dashboard named `Analytics basics (wizard)` to hold the -views for the events this integration instrumented. Keep the `(wizard)` tag with -that exact casing so a search for `(wizard)` surfaces every wizard-created -artifact. - -Everything here goes through `posthog_exec` (`call `). - -## Create the dashboard - -Create the parent dashboard first with `dashboard-create`, and capture its -returned `id` — the insight step attaches every insight to it via -`dashboards: []`: - -```json -{ "name": "Analytics basics (wizard)", "description": "Key views for the events instrumented by the PostHog wizard.", "tags": ["wizard"] } -``` - -Then create the insights (see the insight step) and attach each one to this -dashboard's `id`. Always create the dashboard and its insights based on the -intended captures, regardless of whether those events have been observed yet — -an empty dashboard shell is not a success. Fresh insights render empty and fill -in as events arrive; that is the expected state of a brand-new integration, not -a reason to hold back. - -## Hand off the dashboard - -Emit the dashboard URL on its own line in your final message with this exact -marker so the wizard surfaces it: `[DASHBOARD_URL] `. A URL only -in prose, without the marker, is dropped. Also record the dashboard URL in your -handoff so the report step can link it. diff --git a/context/skills/integration-v2/enable-features/config.yaml b/context/skills/integration-v2/enable-features/config.yaml new file mode 100644 index 00000000..6db5d403 --- /dev/null +++ b/context/skills/integration-v2/enable-features/config.yaml @@ -0,0 +1,12 @@ +# Orchestrator step-skill: fetched by task agents, never surfaced as a command. +type: docs-only +template: description.md +description: Turn on the PostHog products self-driving reads from +tags: [orchestrator, enable-features] +variants: + - id: all + display_name: PostHog enable-features step + tags: [orchestrator, enable-features] + docs_urls: [] +cli: + role: internal diff --git a/context/skills/integration-v2/enable-features/description.md b/context/skills/integration-v2/enable-features/description.md new file mode 100644 index 00000000..d4c08a95 --- /dev/null +++ b/context/skills/integration-v2/enable-features/description.md @@ -0,0 +1,28 @@ +# Enable the PostHog products + +The SDK is in and initialized. These are the project-level switches that decide +whether the data it sends is actually collected — a flip in PostHog, not a change +in the repo. + +{{> enable-products}} + +## You don't touch code + +The init step of this run owns the client's init options, so the contradicting +options above are already its business, not yours. If you find one that cancels a +flip you just made, name the file and the option in your handoff — review fixes it. +Never edit the repo from here. + +## Reading the product signals + +The `product-signals` step read the repo for evidence about each product, and you are +queued behind it, so its handoff is already in your context as one line per product. If +it somehow isn't, call `read_handoffs({ type: 'product-signals' })` once — then enable +the products anyway and say in your handoff that you worked without signals. Never wait +and never poll. + +Where the signals mark a product `partial`, PostHog would overlap something the +project already runs (an existing error reporter, a logger already shipping +elsewhere). Still enable it — overlap is the user's call to make, not yours — but +name the overlap and the file it came from in your handoff so the report can offer +the user the choice. diff --git a/context/skills/integration-v2/init/description.md b/context/skills/integration-v2/init/description.md index ab3681f2..13975b9f 100644 --- a/context/skills/integration-v2/init/description.md +++ b/context/skills/integration-v2/init/description.md @@ -39,6 +39,26 @@ Follow the reference example and the docs for this framework's pattern. Read the existing provider, entry, or startup file before editing, and add PostHog alongside what is already there rather than replacing it. +## The products that are init options + +Product analytics, web analytics, and session replay are not separate steps of this +run — they are what a correctly configured init already gives you. For a browser SDK +that means leaving the defaults alone: autocapture and pageview capture are on unless +the init switches them off, and pageleave is what turns pageviews into web-analytics +sessions. Do not pass an option to re-assert a default. + +What matters is not *cancelling* them. If the project's existing init, or an example +you are adapting, carries `autocapture: false`, `capture_pageview: false`, or +`disable_session_recording: true`, that option is what decides the product is off, and +no project-side switch can overrule it. Leave a deliberate choice the project already +made in place — but name in your handoff which of these options are set and where. A +later step turns these products on in PostHog and needs to know what the client does +with them. + +Server and mobile SDKs have no equivalent: none of this is an init option there, and +replay or exception capture on those platforms needs its own setup later. Say that +rather than reaching for an option that doesn't exist. + ## Content Security Policy Before wiring a browser SDK, check whether the app ships a CSP — a meta tag in diff --git a/context/skills/integration-v2/insight/config.yaml b/context/skills/integration-v2/insight/config.yaml deleted file mode 100644 index c367b9e2..00000000 --- a/context/skills/integration-v2/insight/config.yaml +++ /dev/null @@ -1,11 +0,0 @@ -type: docs-only -template: description.md -description: Create PostHog insights (trends, funnels) attached to a dashboard -tags: [orchestrator, insight] -variants: - - id: all - display_name: PostHog insight step - tags: [orchestrator, insight] - docs_urls: [] -cli: - role: internal diff --git a/context/skills/integration-v2/insight/description.md b/context/skills/integration-v2/insight/description.md deleted file mode 100644 index a19f6c11..00000000 --- a/context/skills/integration-v2/insight/description.md +++ /dev/null @@ -1,64 +0,0 @@ -# Create insights - -Populate the dashboard with up to five insights from the events this integration -instrumented — lead with the business-critical views: conversion funnels, -activation, and churn signals. Use the exact event names from the capture step's -handoff; never invent events that were not instrumented. Keep each name tagged -`(wizard)` with that exact casing. - -Always create these insights based on the intended captures, regardless of -whether those events have been observed yet. An insight is a definition over -event names, not a snapshot of current data: it is expected to render empty -until the first events arrive, and it fills in on its own once they do. "No -data ingested yet", "the events aren't in the schema", or "the query would -return nothing today" are never reasons to skip or defer insights — a dashboard -handed off without them is an incomplete integration, not a cautious one. - -Every call goes through `posthog_exec` (`call insight-create `). Attach each -insight to the dashboard by passing its id in `dashboards: []`. - -Use these known-good query shapes — they are verified against the MCP schema, and -the common variations around them are rejected. - -A trends insight with a breakdown (breakdowns go in `breakdownFilter.breakdowns`, -an array — there is NO top-level `breakdown` field on `TrendsQuery`): - -```json -{ - "name": "Signups by plan (wizard)", - "dashboards": [], - "query": { "kind": "InsightVizNode", "source": { - "kind": "TrendsQuery", - "series": [{ "kind": "EventsNode", "event": "user_signed_up", "math": "total" }], - "interval": "day", - "dateRange": { "date_from": "-30d" }, - "breakdownFilter": { "breakdowns": [{ "type": "event", "property": "plan" }] }, - "trendsFilter": { "display": "ActionsBar" } - }} -} -``` - -A conversion funnel (window fields are camelCase and live INSIDE `funnelsFilter`, -not at the top level of `FunnelsQuery`, and not snake_case): - -```json -{ - "name": "Signup funnel (wizard)", - "dashboards": [], - "query": { "kind": "InsightVizNode", "source": { - "kind": "FunnelsQuery", - "series": [ - { "kind": "EventsNode", "event": "page_viewed" }, - { "kind": "EventsNode", "event": "user_signed_up" } - ], - "dateRange": { "date_from": "-30d" }, - "funnelsFilter": { "funnelVizType": "steps", "funnelOrderType": "ordered", "funnelWindowInterval": 14, "funnelWindowIntervalUnit": "day" } - }} -} -``` - -Valid `trendsFilter.display` values: `ActionsLineGraph`, `ActionsBar`, -`ActionsAreaGraph`, `ActionsPie`, `ActionsStackedBar`, `BoldNumber`, -`ActionsTable`. Names like `ActionsBarChart`/`ActionsBarGraph` are rejected. If an -insight call is rejected, fix the payload against these examples rather than -retrying variations. diff --git a/context/skills/integration-v2/report/description.md b/context/skills/integration-v2/report/description.md index 1652e80e..60c942ec 100644 --- a/context/skills/integration-v2/report/description.md +++ b/context/skills/integration-v2/report/description.md @@ -1,49 +1,69 @@ # Write the setup report -Write `posthog-setup-report.md` at the project root summarizing the integration. -When the file doesn't exist, write it directly — don't attempt a read first. If a -previous run left one behind, `Read` it, then replace it wholesale (harnesses -refuse to overwrite a file that wasn't read; nothing in the old report is worth -merging). -Draw on two sources only: +Write `posthog-setup-report.md` at the project root. When the file doesn't exist, write +it directly — don't attempt a read first. If a previous run left one behind, `Read` it, +then replace it wholesale (harnesses refuse to overwrite a file that wasn't read; +nothing in the old report is worth merging). -- the run's queue log — `.posthog-wizard-cache/queue.json`, which holds each - task's handoff inline — for what each step did, whether identify was wired - or skipped, and any build conflict; -- `.posthog-wizard-cache/.posthog-events.json` — the events that were instrumented. - If that file is missing or empty, reconstruct the list instead of dropping the - table: grep the changed files for `capture(` calls and read the capture step's - handoff in `queue.json`. +One source: the run's queue log, `.posthog-wizard-cache/queue.json`, which holds every +task's handoff inline. It tells you what each step did, whether identify was wired or +skipped, which products were enabled or refused, and any build conflict — and it carries +the `product-signals` handoff, whose `forNextAgent` block is one line per product in the +shape `: yes|no|partial — ()`. That block is what the suggestion +sections are built from. -Include: +This run captured no events, created no insights, and built no dashboard. Do not write a +table of instrumented events, and do not link a dashboard — there isn't one. -- A one-line summary of what was set up. -- What was installed and how PostHog was initialized. -- The events instrumented, as a table: event name, what it measures, and the file - (from `.posthog-wizard-cache/.posthog-events.json`). -- Whether user identification was wired or skipped, and why. -- The error tracking added. -- The dashboard link. -- Any build conflict, in full. -- Clear next steps for the user. +## Sections, in this order -End with a "Before you merge" checklist as GitHub-style checkboxes (`- [ ] …`), -including only the items that apply to what was set up — judge each against the -code changed this run and drop the ones that don't fit: +1. **What's set up.** One line, then a table of reality only: the SDK and its version, + the init point (file), identify (file, or skipped and why), error tracking (mechanism + and file), the products enabled in the project with their per-product result, and LLM + instrumentation if that step ran. +2. **What we left for your agent.** State plainly that no events, funnels, insights, or + dashboards were created, and why: those choices belong to whoever knows the product, + and the agent working in this repo is better placed to make them than a setup run is. +3. **Suggested events.** A table of event name, what it would measure, and the file to + instrument it in. Draw them from the `product-analytics` signal line and the routes + and entry points it names. Names and locations only — no code. +4. **What those events unlock.** The insights, funnels, and retention questions those + events make answerable, each naming the events it needs. Keep to analyses the + suggested events actually support. +5. **Driving PostHog from your agent.** How to reach the PostHog MCP, the tools that + matter for this work (`insight-create`, `dashboard-create`, `query-run`), and the + PostHog skills already on disk in this repo, by path. +6. **Prompts to paste into your agent.** Literal blockquoted prompts, one per follow-up: + instrument the suggested events; build a dashboard from them; connect a data source; + wire source-map upload if the app ships minified bundles. Each names the skill it + leans on and the files it should start from. +7. **Suggested data sources.** From the `data-sources` and `revenue` signal lines: what + was found, and how to connect each one. +8. **Before you merge.** GitHub-style checkboxes (`- [ ] …`), only the items that apply to + what was set up — judge each against the code changed this run and drop the rest: + - Always: run a full production build (the run only verified the files it touched) and + fix any lint or type errors the generated code introduced. + - Always: run the test suite — new init and instrumentation may need updated mocks or + fixtures. + - If env vars were added: their exact names are in `.env.example` and any + monorepo/bootstrap scripts, and set in the deploy environments, not just locally. + - If the app ships minified browser bundles: wire source-map upload into CI so + production stack traces de-minify — call it out with the docs link. + - If the app ships a Content-Security-Policy: load the app and check the console for + CSP violations — a blocked SDK queues events silently and never sends. + - If LLM analytics was set up: trigger the instrumented call path and confirm + `$ai_generation` events appear in PostHog. + - If auth exists and identify was wired: the returning-visitor path also calls + identify, so returning sessions don't fragment onto anonymous distinct IDs. +9. **What we didn't do, and why.** Every task that ended `not needed` or `failed`, with + its reason, any `conflict` line in full, and every follow-up a handoff raised — a + product that needs project admin, a platform where replay needs its own SDK setup, a + Support inbox with no channel connected yet. Never silent. -- Always: run a full production build (the wizard only verified the files it - touched) and fix any lint or type errors the generated code introduced. -- Always: run the test suite — instrumented call sites may need updated mocks or - fixtures. -- If env vars were added: their exact names are in `.env.example` and any - monorepo/bootstrap scripts, and set in the deploy environments, not just locally. -- If the app ships minified browser bundles: wire source-map upload into CI so - production stack traces de-minify — call it out with the docs link. -- If the app ships a Content-Security-Policy: load the app and check the console - for CSP violations — a blocked SDK queues events silently and never sends. -- If LLM analytics was set up: trigger the instrumented call path and confirm - `$ai_generation` events appear in PostHog. -- If auth exists and identify was wired: the returning-visitor path also calls - identify, so returning sessions don't fragment onto anonymous distinct IDs. +Sections 3, 4, and 7 exist only because the groom ran. If there is no `product-signals` +handoff in the queue log, keep the headings and say the groom didn't complete, so the +absence is visible rather than papered over. The same rule holds for any step whose +handoff is missing — name the step and say its outcome is unknown rather than omitting +it or assuming it worked. -Keep it skimmable. This is the artifact the user opens after the run. +Keep it skimmable. This is the artifact the user opens, and the brief their agent reads. diff --git a/context/skills/self-driving/references/3b-enable-products.md b/context/skills/self-driving/references/3b-enable-products.md index 119a76a6..68117d29 100644 --- a/context/skills/self-driving/references/3b-enable-products.md +++ b/context/skills/self-driving/references/3b-enable-products.md @@ -4,9 +4,7 @@ next_step: 4-sources.md # Step 3b — Enable products -Turn ON the PostHog products that Signals reads from — **Session Replay**, **Error Tracking**, and **Support** (Conversations) — so the sources you enable in the next step have data to read. A source with its product switched off just sits idle. This is a server-side flip with conservative defaults the server owns (you don't pass any settings); enabling Support also mints its widget token but leaves the widget itself off until a channel is connected. - -This is distinct from step 4: here you turn the *products* on; step 4 wires up the *signal sources* that consume them. +This is distinct from step 4: here you turn the *products* on; step 4 wires up the *signal sources* that consume them. **A rejection here does not block the next step** — enabling a product (this step) and enabling its signal source (step 4) are independent calls, so step 4 still switches the sources on. They simply sit idle until the products are on, then pick up data with no re-setup. ## Status @@ -16,30 +14,8 @@ Emit: [STATUS] Enabling products ``` -## Tools - -Reach `products-enable` through the PostHog `exec` tool (`info products-enable`, then `call products-enable `). - ## Do -1. Call `products-enable` to turn the products on: - -``` -{ "products": ["session_replay", "error_tracking", "conversations"] } -``` - - It is idempotent and server-owned — the response is `{ "results": { : "enabled" | "already_enabled" } }`. The run prompt's "Project state read at auth time" block tells you which are already ON, so you can leave those out (re-sending is harmless either way). Record the per-product result — the report lists it. - - If the call is rejected for permissions (e.g. some of these need project admin the user lacks), don't abort: record a follow-up to enable them from a project-admin account, and continue. **A rejection here does not block the next step** — enabling a product (this step) and enabling its signal source (step 4) are independent calls, so step 4 still switches the sources on. They simply sit idle until the products are on, then pick up data with no re-setup. - -2. **Web app** (this repo serves a browser frontend / loads `posthog-js`): the server flip only takes effect if the client init doesn't override it. Find the `posthog.init(...)` call and check its options: - - `disable_session_recording: true` cancels the replay enable → remove that option (or set it `false`). - - `capture_exceptions: false` cancels the error-tracking enable → remove it (or set it `true`). - - If neither is set, the server flip is enough — leave the init alone. - - If you can't confidently locate or edit the init, don't guess — record a follow-up to check it manually. - -3. **Pure backend or mobile app** (no `posthog-js` reads the server config): the flip is inert until the SDK is configured in code. Don't edit code here — record a follow-up noting that replay / exception capture for this platform needs SDK changes (a later setup will handle it). - -4. **Support / Conversations** is now enabled, but it only produces tickets once an inbound channel (email / inbox / Slack) is connected. You're not connecting a channel here — just record it so step 7's report can give the user that next step. +{{> enable-products}} -Record every result and any follow-up — the report needs them. Then continue to the next step. +Step 7's report lists the per-product result and every follow-up, so record them as you go. Then continue to the next step.