Skip to content

Commit fb08094

Browse files
authored
docs(agents): align agent guidance with code and enforced checks (#8887)
* docs(agents): align agent guidance with code and enforced checks - contracts: export the contract; export a schema or type only when imported - testing: mock-assertion ban matches test-audit; .dom.test convention; runner commands - imports: biome organizeImports owns order - stores: devtools for new stores; reset() + registerUserDataReset - add apps/sim/CLAUDE.md and (landing)/AGENTS.md symlinks; name db-migrate skill - skills: safeUrlPathSegment for path segments, hmacSha256Hex and bounded reads in trigger templates, hosted-key per_request/enabledWhen, stale identifiers fixed, pressure markers removed - CONTRIBUTING: commit types and format match history * docs(add-model): keep the check:/generate: script names from staging * docs(emcn-design-review): list only the Chip variants its props accept
1 parent 6835a54 commit fb08094

40 files changed

Lines changed: 112 additions & 114 deletions

File tree

‎.agents/skills/add-block/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -615,7 +615,7 @@ tools: {
615615

616616
## Outputs Definition
617617

618-
**IMPORTANT:** Block outputs have a simpler schema than tool outputs. Block outputs do NOT support:
618+
Block outputs have a simpler schema than tool outputs. They do not support:
619619
- `optional: true` - This is only for tool outputs
620620
- `items` property - This is only for tool outputs with array types
621621

@@ -729,7 +729,7 @@ export const ServiceBlock: BlockConfig = {
729729
longDescription: 'Full description for documentation...',
730730
docsLink: 'https://docs.sim.ai/integrations/service',
731731
category: 'tools',
732-
integrationType: IntegrationType.DeveloperTools,
732+
integrationType: IntegrationType.DevOps,
733733
bgColor: '#FF6B6B',
734734
icon: ServiceIcon,
735735
authMode: AuthMode.OAuth,

‎.agents/skills/add-column-type/SKILL.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ The three that are easy to get wrong:
9898

9999
Add the entry to `COLUMN_TYPE_REGISTRY` in `registry.ts` **and** `COLUMN_TYPE_SERVER_REGISTRY` in `registry.server.ts`.
100100

101-
`COLUMN_TYPES` is declared in `types.ts` (not derived from the registry — the registry is annotated `Record<ColumnType, …>` against it, which is the gate). `constants.ts` re-exports it, so `columnTypeSchema = z.enum(COLUMN_TYPES)` picks your type up with no edit. **Type-specific metadata does not** — see the next step.
101+
`COLUMN_TYPES` is declared in `types.ts` (not derived from the registry — the registry is annotated `Record<ColumnType, …>` against it, which is the gate). `columnTypeSchema` in `lib/api/contracts/tables.ts` is `z.enum(COLUMN_TYPES)`, so it picks your type up with no edit. **Type-specific metadata does not** — see the next step.
102102

103103
## Step 5: Migrations (only if the stored bytes change)
104104

@@ -131,6 +131,7 @@ Registering the *type* is compiler-enforced. Registering its *metadata* is not,
131131
| `lib/table/types.ts` `ColumnDefinition` | (this one DOES fail — the ownership loop indexes it) |
132132
| `column-types/types.ts` `TYPE_SPECIFIC_COLUMN_KEYS` | it is never stripped on conversion, and poisons the target type |
133133
| `lib/api/contracts/tables.ts` — the schema slot in all three column schemas, plus `refineColumnOptions` | zod strips it at the boundary; silently never saved |
134+
| `lib/api/contracts/v2/tables.ts` and `lib/table/application/columns.ts` — the same slots for the v2 API and its use cases | the v2 API silently drops it |
134135
| `columns/service.ts` `addTableColumn` param type | callers cannot pass it |
135136
| A metadata-only update in `lib/table/columns/service.ts` (`updateColumnCurrency` is the model) + a branch in `performUpdateTableColumn` in `lib/table/orchestration/columns.ts` | changing it on an existing column is a silent 200 no-op |
136137
| `column-config-sidebar.tsx` | no UI to set it |

‎.agents/skills/add-connector/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ export const {service}ConnectorMeta: ConnectorMeta = {
8989

9090
configFields: [
9191
// Rendered dynamically by the add-connector modal UI
92-
// Supports 'short-input', 'dropdown', and 'selector' types — see ConfigField Types below
92+
// Supports 'short-input', 'dropdown', and 'selector' types — see ConnectorConfigField Types below
9393
],
9494

9595
// Optional: tag definitions are metadata too — declare them here
@@ -167,7 +167,7 @@ export const {service}Connector: ConnectorConfig = {
167167
}
168168
```
169169

170-
## ConfigField Types
170+
## ConnectorConfigField Types
171171

172172
The add-connector modal renders these automatically — no custom UI needed.
173173

‎.agents/skills/add-enrichment/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ For each output the enrichment produces, decide which existing tool provides it.
3737
- Its `params` accept what you can derive from table columns (read the tool's `params`).
3838
- Its `outputs` / `transformResponse` actually expose the field you need (read the real output shape — don't assume).
3939

40-
Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo / LinkedIn are not hosted-safe (ToS) — don't use them.
40+
Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo and LinkedIn APIs are not hosted-safe (ToS) — never call them as providers.
4141

4242
## Step 2: Verify hosted-key support — chain to `/add-hosted-key` if missing
4343

@@ -109,7 +109,7 @@ export { myEnrichment } from './my-enrichment'
109109
```
110110

111111
Rules:
112-
- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
112+
- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, `@/enrichments/provider-failures/*`, and the types. **Never import `@/tools`** here — the runner does the tool call.
113113
- `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs).
114114
- Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config).
115115

‎.agents/skills/add-hosted-key/SKILL.md‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,20 @@ pricing: {
154154

155155
**`getCost` must always throw** if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.
156156

157+
**When the provider charges a flat price per call** — use `per_request` instead of `getCost` (as `tools/brandfetch/get_brand.ts` does):
158+
159+
```typescript
160+
pricing: {
161+
type: 'per_request',
162+
// $0.04 per call — from https://example.com/pricing
163+
cost: 0.04,
164+
},
165+
```
166+
167+
### Hosted Keys for Some Parameter Combinations
168+
169+
When only some calls can use the hosted key (for example, one provider of several), gate the config with `enabled: hostedKeyEnabledWhen<Params>({ field: 'provider', operator: 'equals', value: 'falai' })` from `@/tools/hosting` (`operator: 'one_of'` takes `values`); `tools/image/generate.ts` is the reference.
170+
157171
### Capturing Cost Data from the API
158172

159173
If the API returns cost info, capture it in `transformResponse` so `getCost` can read it from the output:

‎.agents/skills/add-model/SKILL.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,13 +49,13 @@ Use a precise WebFetch prompt: *"Extract for {model_id}: exact model id string,
4949
|---|---|---|
5050
| `temperature` | All providers (passed through if set) | Safe but inert on always-reasoning models that reject it |
5151
| `toolUsageControl` | All providers (provider-level default) | Override per model only when that model differs |
52-
| `forcedToolUse` | `anthropic/core.ts` (anthropic, azure-anthropic, kie); defaults to `toolUsageControl` | Ignored by every other provider; set `false` only on a model behind that core that cannot force tools |
52+
| `forcedToolUse` | `anthropic/core.ts` (anthropic, azure-anthropic, kie) defaults it to `toolUsageControl` and reads `thinking.forcedToolUse` for forcing while thinking; `openai/core.ts` and `bedrock/index.ts` treat only an explicit `false` as "cannot force" | Ignored by every other provider; set `false` only on a model that cannot force tools |
5353
| `promptCaching` | Caller-placed cache breakpoints | Set only where the vendor charges for opt-in caching (absent for OpenAI/Gemini implicit caching) |
5454
| `reasoningEffort` | `openai/core.ts`, `azure-openai`, `xai`, `deepseek`, `groq`, `zai`, `kimi`, `cerebras`, `meta`, `litellm` (each `index.ts`) | Not read by anthropic/gemini (they use `thinking`) or by mistral, openrouter, fireworks, vertex — re-grep before assuming |
5555
| `verbosity` | `openai/core.ts`, `azure-openai/index.ts` only | Dead elsewhere |
5656
| `thinking` | `anthropic/core.ts`, `gemini/core.ts`; `deepseek`, `groq`, `zai`, `kimi` (each `index.ts`) read the resolved `thinkingLevel` | Dead elsewhere |
5757
| `thinking.streamed` | Docs generator + `getThinkingStreamVisibility` (`models.ts`); `anthropic/core.ts` uses `'summary'` to request `display: 'summarized'` on agent-events runs | **Mandatory on Anthropic-family thinking models** (`check:agent-stream-docs` fails without it); other families fall back to provider defaults |
58-
| `nativeStructuredOutputs` | `anthropic/core.ts`, `bedrock/index.ts` (via `models.ts` `supportsNativeStructuredOutputs`, which reads the flag) | Dead elsewhere — fireworks/baseten/together/openrouter call their own provider-level `supportsNativeStructuredOutputs` that ignores the model flag (always on, always off, or OpenRouter API metadata) |
58+
| `nativeStructuredOutputs` | `anthropic/core.ts`, `bedrock/index.ts` (via `models.ts` `supportsNativeStructuredOutputs`, which reads the flag), `nebius/index.ts`, `nvidia/index.ts` (via `getModelCapabilities`) | Dead elsewhere — fireworks/baseten/together/openrouter call their own provider-level `supportsNativeStructuredOutputs` that ignores the model flag (always on, always off, or OpenRouter API metadata) |
5959
| `maxOutputTokens` | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
6060
| `computerUse` | `providers/utils.ts` (`getComputerUseModels` → `computerUseModels` routing) | Set only on actual computer-use SKUs |
6161
| `deepResearch` | UI flag for routing to deep-research SKUs | Set only on actual deep-research model IDs |

‎.agents/skills/add-tools/SKILL.md‎

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -67,15 +67,12 @@ case with trusted execution context; use the `migrate-application-operation` ski
6767
Use this structure only for an absolute external provider API:
6868

6969
```typescript
70-
import type { {ServiceName}{Action}Params } from '@/tools/{service}/types'
70+
import type {
71+
{ServiceName}{Action}Params,
72+
{ServiceName}{Action}Response,
73+
} from '@/tools/{service}/types'
7174
import type { ToolConfig } from '@/tools/types'
72-
73-
interface {ServiceName}{Action}Response {
74-
success: boolean
75-
output: {
76-
// Define output structure here
77-
}
78-
}
75+
import { safeUrlPathSegment } from '@/tools/url-path'
7976

8077
export const {serviceName}{Action}Tool: ToolConfig<
8178
{ServiceName}{Action}Params,
@@ -117,7 +114,8 @@ export const {serviceName}{Action}Tool: ToolConfig<
117114
},
118115

119116
request: {
120-
url: (params) => `https://api.service.com/v1/resource/${params.id}`,
117+
url: (params) =>
118+
`https://api.service.com/v1/resource/${safeUrlPathSegment(params.someId, 'someId')}`,
121119
method: 'POST',
122120
headers: (params) => ({
123121
Authorization: `Bearer ${params.accessToken}`,
@@ -190,7 +188,7 @@ fallback, or caller-controlled `_context` authority.
190188

191189
A required `'hidden'` param needs an `oauth` declaration or `hosting.apiKeyParam` to supply it (`bun run check:tool-param-reachability`).
192190

193-
A declared `timeout` param is an ordinary tool input — put it in the request body or URL yourself if the provider expects it; it becomes Sim's millisecond request deadline only when the tool sets `timeoutParamIsDeadline: true` (`http_request`). A `method` param on a tool with a fixed `request.method` would be sent as the HTTP verb, so the same audit rejects it.
191+
A declared `timeout` param is an ordinary tool input — put it in the request body or URL yourself if the provider expects it; it becomes Sim's millisecond request deadline only when the tool sets `timeoutParamIsDeadline: true` (e.g. `http_request`). A `method` param on a tool with a fixed `request.method` would be sent as the HTTP verb, so the same audit rejects it.
194192

195193
### Parameter Types
196194
- `'string'` - Text values
@@ -362,7 +360,7 @@ Only use bare `type: 'json'` without `properties` when the shape is truly dynami
362360
## Critical Rules for transformResponse
363361

364362
### Handle Nullable Fields
365-
ALWAYS use `?? null` for fields that may be undefined:
363+
Use `?? null` for fields that may be undefined:
366364
```typescript
367365
transformResponse: async (response: Response) => {
368366
const data = await response.json()
@@ -465,7 +463,7 @@ these are regenerated — and CI fails on stale artifacts. Commit the result. Se
465463

466464
## Wiring Tools into the Block (Required)
467465

468-
After registering in `tools/registry.ts`, you MUST also update the block definition at `apps/sim/blocks/blocks/{service}.ts`. This is not optional — tools are only usable from the UI if they are wired into the block.
466+
After registering in `tools/registry.ts`, also update the block definition at `apps/sim/blocks/blocks/{service}.ts`: a tool is usable from the UI only once the block wires it.
469467

470468
### 1. Add to `tools.access`
471469

‎.agents/skills/add-trigger/SKILL.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -216,18 +216,17 @@ If none apply, you don't need a handler. The default handler provides bearer tok
216216
### Example Handler
217217

218218
```typescript
219-
import crypto from 'crypto'
220219
import { createLogger } from '@sim/logger'
221220
import { safeCompare } from '@sim/security/compare'
221+
import { hmacSha256Hex } from '@sim/security/hmac'
222222
import type { EventMatchContext, FormatInputContext, FormatInputResult, WebhookProviderHandler } from '@/lib/webhooks/providers/types'
223223
import { createHmacVerifier } from '@/lib/webhooks/providers/utils'
224224

225225
const logger = createLogger('WebhookProvider:{Service}')
226226

227227
function validate{Service}Signature(secret: string, signature: string, body: string): boolean {
228228
if (!secret || !signature || !body) return false
229-
const computed = crypto.createHmac('sha256', secret).update(body, 'utf8').digest('hex')
230-
return safeCompare(computed, signature)
229+
return safeCompare(hmacSha256Hex(body, secret), signature)
231230
}
232231

233232
export const {service}Handler: WebhookProviderHandler = {
@@ -299,6 +298,7 @@ If they differ: the tag dropdown shows fields that don't exist, or actual data h
299298
If the service API supports programmatic webhook creation, implement `createSubscription` and `deleteSubscription` on the handler. The orchestration layer calls these automatically — **no code touches `route.ts`, `provider-subscriptions.ts`, or `deploy.ts`**.
300299

301300
```typescript
301+
import { readResponseJsonWithLimit } from '@/lib/core/utils/stream-limits'
302302
import { getNotificationUrl, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils'
303303
import type { DeleteSubscriptionContext, SubscriptionContext, SubscriptionResult } from '@/lib/webhooks/providers/types'
304304

@@ -315,7 +315,10 @@ export const {service}Handler: WebhookProviderHandler = {
315315
})
316316

317317
if (!res.ok) throw new Error(`{Service} error: ${res.status}`)
318-
const { id } = (await res.json()) as { id: string }
318+
const { id } = await readResponseJsonWithLimit<{ id: string }>(res, {
319+
maxBytes: 1024 * 1024,
320+
label: '{Service} webhook creation response',
321+
})
319322
return { providerConfigUpdates: { externalId: id } }
320323
},
321324

@@ -342,6 +345,7 @@ export const {service}Handler: WebhookProviderHandler = {
342345
Trigger outputs use the same schema as block outputs (NOT tool outputs).
343346

344347
**Supported:** `type` + `description` for leaf fields, nested objects for complex data.
348+
**Also supported:** `nullable: true` and `condition` (`TriggerOutput` in `triggers/types.ts`).
345349
**NOT supported:** `optional: true`, `items` (those are tool-output-only features).
346350

347351
```typescript
@@ -377,15 +381,15 @@ apps/sim/lib/webhooks/polling/
377381

378382
```typescript
379383
import { pollingIdempotency } from '@/lib/core/idempotency/service'
380-
import type { PollingProviderHandler, PollWebhookContext } from '@/lib/webhooks/polling/types'
384+
import type { PollingProviderHandler, PollOutcome, PollWebhookContext } from '@/lib/webhooks/polling/types'
381385
import { markWebhookFailed, markWebhookSuccess, resolveOAuthCredential, updateWebhookProviderConfig } from '@/lib/webhooks/polling/utils'
382386
import { processPolledWebhookEvent } from '@/lib/webhooks/processor'
383387

384388
export const {service}PollingHandler: PollingProviderHandler = {
385389
provider: '{service}',
386390
label: '{Service}',
387391

388-
async pollWebhook(ctx: PollWebhookContext): Promise<'success' | 'failure'> {
392+
async pollWebhook(ctx: PollWebhookContext): Promise<PollOutcome> {
389393
const { webhookData, workflowData, requestId, logger } = ctx
390394
const webhookId = webhookData.id
391395

‎.agents/skills/council/SKILL.md‎

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,6 @@
22
name: council
33
description: Spawn parallel task agents to explore a given area of the codebase from multiple angles, then use their findings to answer the question or build a plan. Use when a task needs broad fan-out exploration across many files before acting.
44
argument-hint: <area-of-interest>
5-
# No agents/openai.yaml by design: council is a meta/exploration utility (like cleanup, ship, you-might-not-need-*), not a service-integration builder, so it intentionally ships no standalone agent card.
65
---
76

8-
Based on the given area of interest, please:
9-
10-
1. Dig around the codebase in terms of that given area of interest, gather general information such as keywords and architecture overview.
11-
2. Spawn off n=10 (unless specified otherwise) task agents to dig deeper into the codebase in terms of that given area of interest, some of them should be out of the box for variance.
12-
3. Once the task agents are done, use the information to do what the user wants.
13-
14-
If user is in plan mode, use the information to create the plan.
7+
Map the area of interest first (keywords, architecture), then fan out parallel agents, each exploring a distinct angle, including a few unconventional ones. Size the fan-out to the area (the user may name a number). Use their findings to answer the question, or to write the plan in plan mode.

‎.agents/skills/db-migrate/SKILL.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Never put expand and contract in the same PR. If this PR both removes the code t
3333
| Drop a column/table | stop all reads/writes in code; ship it | `DROP` (annotate) |
3434
| Change a column type | add a new column of the new type; dual-write | backfill, swap reads, drop old |
3535
| Add FK / CHECK | `ADD CONSTRAINT ... NOT VALID` | `VALIDATE CONSTRAINT` separately |
36-
| Index an existing table | `COMMIT;` breakpoint → `SET lock_timeout = 0` → `CREATE INDEX CONCURRENTLY IF NOT EXISTS` (see `packages/db/scripts/migrate.ts`) | — |
36+
| Index an existing table | `COMMIT;` breakpoint → `SET lock_timeout = 0` → `CREATE INDEX CONCURRENTLY IF NOT EXISTS` → `SET lock_timeout = '5s'` (see `packages/db/scripts/migrate.ts`) | — |
3737
| Drop an index | `COMMIT;` breakpoint → `DROP INDEX CONCURRENTLY IF EXISTS` — plain `DROP INDEX` takes ACCESS EXCLUSIVE on the table | — |
3838
| Backfill data | batched + idempotent `UPDATE` (keyset/`WHERE`, bounded) | — |
3939

0 commit comments

Comments
 (0)