Merge remote-tracking branch 'origin/staging' into chore/test-patterns-ratchet

# Conflicts:
#	.claude/rules/sim-testing.md
#	.cursor/rules/sim-testing.mdc

Author: waleedlatif1

.agents/skills/add-block/SKILL.md

@@ -615,7 +615,7 @@ tools: {
 
 ## Outputs Definition
 
-**IMPORTANT:** Block outputs have a simpler schema than tool outputs. Block outputs do NOT support:
+Block outputs have a simpler schema than tool outputs. They do not support:
 - `optional: true` - This is only for tool outputs
 - `items` property - This is only for tool outputs with array types
 
@@ -729,7 +729,7 @@ export const ServiceBlock: BlockConfig = {
   longDescription: 'Full description for documentation...',
   docsLink: 'https://docs.sim.ai/integrations/service',
   category: 'tools',
-  integrationType: IntegrationType.DeveloperTools,
+  integrationType: IntegrationType.DevOps,
   bgColor: '#FF6B6B',
   icon: ServiceIcon,
   authMode: AuthMode.OAuth,

.agents/skills/add-column-type/SKILL.md

@@ -98,7 +98,7 @@ The three that are easy to get wrong:
 
 Add the entry to `COLUMN_TYPE_REGISTRY` in `registry.ts` **and** `COLUMN_TYPE_SERVER_REGISTRY` in `registry.server.ts`.
 
-`COLUMN_TYPES` is declared in `types.ts` (not derived from the registry — the registry is annotated `Record<ColumnType, …>` against it, which is the gate). `constants.ts` re-exports it, so `columnTypeSchema = z.enum(COLUMN_TYPES)` picks your type up with no edit. **Type-specific metadata does not** — see the next step.
+`COLUMN_TYPES` is declared in `types.ts` (not derived from the registry — the registry is annotated `Record<ColumnType, …>` against it, which is the gate). `columnTypeSchema` in `lib/api/contracts/tables.ts` is `z.enum(COLUMN_TYPES)`, so it picks your type up with no edit. **Type-specific metadata does not** — see the next step.
 
 ## Step 5: Migrations (only if the stored bytes change)
 
@@ -131,6 +131,7 @@ Registering the *type* is compiler-enforced. Registering its *metadata* is not,
 | `lib/table/types.ts` `ColumnDefinition` | (this one DOES fail — the ownership loop indexes it) |
 | `column-types/types.ts` `TYPE_SPECIFIC_COLUMN_KEYS` | it is never stripped on conversion, and poisons the target type |
 | `lib/api/contracts/tables.ts` — the schema slot in all three column schemas, plus `refineColumnOptions` | zod strips it at the boundary; silently never saved |
+| `lib/api/contracts/v2/tables.ts` and `lib/table/application/columns.ts` — the same slots for the v2 API and its use cases | the v2 API silently drops it |
 | `columns/service.ts` `addTableColumn` param type | callers cannot pass it |
 | A metadata-only update in `lib/table/columns/service.ts` (`updateColumnCurrency` is the model) + a branch in `performUpdateTableColumn` in `lib/table/orchestration/columns.ts` | changing it on an existing column is a silent 200 no-op |
 | `column-config-sidebar.tsx` | no UI to set it |

.agents/skills/add-connector/SKILL.md

@@ -89,7 +89,7 @@ export const {service}ConnectorMeta: ConnectorMeta = {
 
   configFields: [
     // Rendered dynamically by the add-connector modal UI
-    // Supports 'short-input', 'dropdown', and 'selector' types — see ConfigField Types below
+    // Supports 'short-input', 'dropdown', and 'selector' types — see ConnectorConfigField Types below
   ],
 
   // Optional: tag definitions are metadata too — declare them here
@@ -167,7 +167,7 @@ export const {service}Connector: ConnectorConfig = {
 }
 ```
 
-## ConfigField Types
+## ConnectorConfigField Types
 
 The add-connector modal renders these automatically — no custom UI needed.
 

.agents/skills/add-enrichment/SKILL.md

@@ -37,7 +37,7 @@ For each output the enrichment produces, decide which existing tool provides it.
 - Its `params` accept what you can derive from table columns (read the tool's `params`).
 - Its `outputs` / `transformResponse` actually expose the field you need (read the real output shape — don't assume).
 
-Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo / LinkedIn are not hosted-safe (ToS) — don't use them.
+Order providers **cheapest / most-likely-to-hit first**; the cascade stops at the first non-empty result. Apollo and LinkedIn APIs are not hosted-safe (ToS) — never call them as providers.
 
 ## Step 2: Verify hosted-key support — chain to `/add-hosted-key` if missing
 
@@ -109,7 +109,7 @@ export { myEnrichment } from './my-enrichment'
 ```
 
 Rules:
-- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, and the types. **Never import `@/tools`** here — the runner does the tool call.
+- Keep the file **client-safe**: import only `@sim/emcn/icons`, `@sim/utils/*`, `@/enrichments/providers`, `@/enrichments/provider-failures/*`, and the types. **Never import `@/tools`** here — the runner does the tool call.
 - `buildParams` returns `null` when inputs are insufficient (provider skipped). `mapOutput` returns `null`/empty for a miss (falls through). Use `filterUndefined` when assembling optional tool params; coerce numbers explicitly (don't pass `''` to number outputs).
 - Output `id`s are the keys `mapOutput` returns; output `name`s are the default column names (the user can rename them in the config).
 

.agents/skills/add-hosted-key/SKILL.md

@@ -154,6 +154,20 @@ pricing: {
 
 **`getCost` must always throw** if it cannot determine cost. Never silently fall back to a default — this would hide billing inaccuracies.
 
+**When the provider charges a flat price per call** — use `per_request` instead of `getCost` (as `tools/brandfetch/get_brand.ts` does):
+
+```typescript
+pricing: {
+  type: 'per_request',
+  // $0.04 per call — from https://example.com/pricing
+  cost: 0.04,
+},
+```
+
+### Hosted Keys for Some Parameter Combinations
+
+When only some calls can use the hosted key (for example, one provider of several), gate the config with `enabled: hostedKeyEnabledWhen<Params>({ field: 'provider', operator: 'equals', value: 'falai' })` from `@/tools/hosting` (`operator: 'one_of'` takes `values`); `tools/image/generate.ts` is the reference.
+
 ### Capturing Cost Data from the API
 
 If the API returns cost info, capture it in `transformResponse` so `getCost` can read it from the output:

.agents/skills/add-model/SKILL.md

@@ -49,13 +49,13 @@ Use a precise WebFetch prompt: *"Extract for {model_id}: exact model id string,
 |---|---|---|
 | `temperature` | All providers (passed through if set) | Safe but inert on always-reasoning models that reject it |
 | `toolUsageControl` | All providers (provider-level default) | Override per model only when that model differs |
-| `forcedToolUse` | `anthropic/core.ts` (anthropic, azure-anthropic, kie); defaults to `toolUsageControl` | Ignored by every other provider; set `false` only on a model behind that core that cannot force tools |
+| `forcedToolUse` | `anthropic/core.ts` (anthropic, azure-anthropic, kie) defaults it to `toolUsageControl` and reads `thinking.forcedToolUse` for forcing while thinking; `openai/core.ts` and `bedrock/index.ts` treat only an explicit `false` as "cannot force" | Ignored by every other provider; set `false` only on a model that cannot force tools |
 | `promptCaching` | Caller-placed cache breakpoints | Set only where the vendor charges for opt-in caching (absent for OpenAI/Gemini implicit caching) |
 | `reasoningEffort` | `openai/core.ts`, `azure-openai`, `xai`, `deepseek`, `groq`, `zai`, `kimi`, `cerebras`, `meta`, `litellm` (each `index.ts`) | Not read by anthropic/gemini (they use `thinking`) or by mistral, openrouter, fireworks, vertex — re-grep before assuming |
 | `verbosity` | `openai/core.ts`, `azure-openai/index.ts` only | Dead elsewhere |
 | `thinking` | `anthropic/core.ts`, `gemini/core.ts`; `deepseek`, `groq`, `zai`, `kimi` (each `index.ts`) read the resolved `thinkingLevel` | Dead elsewhere |
 | `thinking.streamed` | Docs generator + `getThinkingStreamVisibility` (`models.ts`); `anthropic/core.ts` uses `'summary'` to request `display: 'summarized'` on agent-events runs | **Mandatory on Anthropic-family thinking models** (`check:agent-stream-docs` fails without it); other families fall back to provider defaults |
-| `nativeStructuredOutputs` | `anthropic/core.ts`, `bedrock/index.ts` (via `models.ts` `supportsNativeStructuredOutputs`, which reads the flag) | Dead elsewhere — fireworks/baseten/together/openrouter call their own provider-level `supportsNativeStructuredOutputs` that ignores the model flag (always on, always off, or OpenRouter API metadata) |
+| `nativeStructuredOutputs` | `anthropic/core.ts`, `bedrock/index.ts` (via `models.ts` `supportsNativeStructuredOutputs`, which reads the flag), `nebius/index.ts`, `nvidia/index.ts` (via `getModelCapabilities`) | Dead elsewhere — fireworks/baseten/together/openrouter call their own provider-level `supportsNativeStructuredOutputs` that ignores the model flag (always on, always off, or OpenRouter API metadata) |
 | `maxOutputTokens` | Read by UI + executor for token estimation | Always meaningful — set if provider documents a cap |
 | `computerUse` | `providers/utils.ts` (`getComputerUseModels` → `computerUseModels` routing) | Set only on actual computer-use SKUs |
 | `deepResearch` | UI flag for routing to deep-research SKUs | Set only on actual deep-research model IDs |

.agents/skills/add-permission-group-item/SKILL.md

@@ -45,7 +45,7 @@ Allowlist when the safe posture is "only what the admin named" and the member se
 
 **Is the decision knowable from the config alone?** A rule needing a request value (an auth mode, a connector id) is *parameterized* and cannot be declared on an operation — see Step 3.
 
-**Is it a gate or a projection?** A key that withholds *fields from a response* rather than the response is a projection. `hideTraceSpans` and `hideCostInfo` work this way: the logs routes declare `capability: 'none'` and strip fields, because refusing the read would withhold the status and error message too. Projections have one owner — `lib/logs/log-projection.ts` (`resolveLogFieldProjection`, `projectExecutionData`, `projectCostTotal`), carrying the `permission-group-enforced:` annotations. Add yours there; two copies of a redaction rule is how one of them stops redacting. Corollary: refuse the query that *selects on* a withheld field — otherwise the projection is a filter oracle; `logQuerySelectsCost` / `assertLogCostQueryAllowed` in that same module are the shape.
+**Is it a gate or a projection?** A key that withholds *fields from a response* rather than the response is a projection. `hideTraceSpans` and `hideCostInfo` work this way: the logs routes declare `capability: 'none'` and strip fields, because refusing the read would withhold the status and error message too. Projections have one owner — `lib/logs/projection.ts` (`resolveLogFieldProjection`, `projectExecutionData`, `projectCostTotal`), carrying the `permission-group-enforced:` annotations. Add yours there; two copies of a redaction rule is how one of them stops redacting. Corollary: refuse the query that *selects on* a withheld field — otherwise the projection is a filter oracle; `logQuerySelectsCost` / `assertLogCostQueryAllowed` in that same module are the shape.
 
 ## Step 1: Append the field entry — never insert
 
@@ -255,7 +255,7 @@ What a run *does* is still governed by `assertPermissionsAllowed`. An item that
 ## Checklist Before Finishing
 
 - [ ] Kind and `enforcement` chosen deliberately; `ui-only` justified in writing if used
-- [ ] It is a gate, not a projection — a projection belongs in `lib/logs/log-projection.ts` with `capability: 'none'` on the routes, and still refuses queries that select on the withheld field
+- [ ] It is a gate, not a projection — a projection belongs in `lib/logs/projection.ts` with `capability: 'none'` on the routes, and still refuses queries that select on the withheld field
 - [ ] Entry **appended** to `PERMISSION_GROUP_FIELDS`, permissive default, restriction-phrased name
 - [ ] Category present in `PLATFORM_CATEGORY_ORDER`, named after what is withheld
 - [ ] `hint` says what access is revoked, never "hide" — it is also the active-restriction prose

.agents/skills/add-tools/SKILL.md

@@ -67,15 +67,12 @@ case with trusted execution context; use the `migrate-application-operation` ski
 Use this structure only for an absolute external provider API:
 
 ```typescript
-import type { {ServiceName}{Action}Params } from '@/tools/{service}/types'
+import type {
+  {ServiceName}{Action}Params,
+  {ServiceName}{Action}Response,
+} from '@/tools/{service}/types'
 import type { ToolConfig } from '@/tools/types'
-
-interface {ServiceName}{Action}Response {
-  success: boolean
-  output: {
-    // Define output structure here
-  }
-}
+import { safeUrlPathSegment } from '@/tools/url-path'
 
 export const {serviceName}{Action}Tool: ToolConfig<
   {ServiceName}{Action}Params,
@@ -117,7 +114,8 @@ export const {serviceName}{Action}Tool: ToolConfig<
   },
 
   request: {
-    url: (params) => `https://api.service.com/v1/resource/${params.id}`,
+    url: (params) =>
+      `https://api.service.com/v1/resource/${safeUrlPathSegment(params.someId, 'someId')}`,
     method: 'POST',
     headers: (params) => ({
       Authorization: `Bearer ${params.accessToken}`,
@@ -190,7 +188,7 @@ fallback, or caller-controlled `_context` authority.
 
 A required `'hidden'` param needs an `oauth` declaration or `hosting.apiKeyParam` to supply it (`bun run check:tool-param-reachability`).
 
-A declared `timeout` param is an ordinary tool input — put it in the request body or URL yourself if the provider expects it; it becomes Sim's millisecond request deadline only when the tool sets `timeoutParamIsDeadline: true` (`http_request`). A `method` param on a tool with a fixed `request.method` would be sent as the HTTP verb, so the same audit rejects it.
+A declared `timeout` param is an ordinary tool input — put it in the request body or URL yourself if the provider expects it; it becomes Sim's millisecond request deadline only when the tool sets `timeoutParamIsDeadline: true` (e.g. `http_request`). A `method` param on a tool with a fixed `request.method` would be sent as the HTTP verb, so the same audit rejects it.
 
 ### Parameter Types
 - `'string'` - Text values
@@ -362,7 +360,7 @@ Only use bare `type: 'json'` without `properties` when the shape is truly dynami
 ## Critical Rules for transformResponse
 
 ### Handle Nullable Fields
-ALWAYS use `?? null` for fields that may be undefined:
+Use `?? null` for fields that may be undefined:
 ```typescript
 transformResponse: async (response: Response) => {
   const data = await response.json()
@@ -465,7 +463,7 @@ these are regenerated — and CI fails on stale artifacts. Commit the result. Se
 
 ## Wiring Tools into the Block (Required)
 
-After registering in `tools/registry.ts`, you MUST also update the block definition at `apps/sim/blocks/blocks/{service}.ts`. This is not optional — tools are only usable from the UI if they are wired into the block.
+After registering in `tools/registry.ts`, also update the block definition at `apps/sim/blocks/blocks/{service}.ts`: a tool is usable from the UI only once the block wires it.
 
 ### 1. Add to `tools.access`
 

.agents/skills/add-trigger/SKILL.md

@@ -216,18 +216,17 @@ If none apply, you don't need a handler. The default handler provides bearer tok
 ### Example Handler
 
 ```typescript
-import crypto from 'crypto'
 import { createLogger } from '@sim/logger'
 import { safeCompare } from '@sim/security/compare'
+import { hmacSha256Hex } from '@sim/security/hmac'
 import type { EventMatchContext, FormatInputContext, FormatInputResult, WebhookProviderHandler } from '@/lib/webhooks/providers/types'
 import { createHmacVerifier } from '@/lib/webhooks/providers/utils'
 
 const logger = createLogger('WebhookProvider:{Service}')
 
 function validate{Service}Signature(secret: string, signature: string, body: string): boolean {
   if (!secret || !signature || !body) return false
-  const computed = crypto.createHmac('sha256', secret).update(body, 'utf8').digest('hex')
-  return safeCompare(computed, signature)
+  return safeCompare(hmacSha256Hex(body, secret), signature)
 }
 
 export const {service}Handler: WebhookProviderHandler = {
@@ -299,6 +298,7 @@ If they differ: the tag dropdown shows fields that don't exist, or actual data h
 If the service API supports programmatic webhook creation, implement `createSubscription` and `deleteSubscription` on the handler. The orchestration layer calls these automatically — **no code touches `route.ts`, `provider-subscriptions.ts`, or `deploy.ts`**.
 
 ```typescript
+import { readResponseJsonWithLimit } from '@/lib/core/utils/stream-limits'
 import { getNotificationUrl, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils'
 import type { DeleteSubscriptionContext, SubscriptionContext, SubscriptionResult } from '@/lib/webhooks/providers/types'
 
@@ -315,7 +315,10 @@ export const {service}Handler: WebhookProviderHandler = {
     })
 
     if (!res.ok) throw new Error(`{Service} error: ${res.status}`)
-    const { id } = (await res.json()) as { id: string }
+    const { id } = await readResponseJsonWithLimit<{ id: string }>(res, {
+      maxBytes: 1024 * 1024,
+      label: '{Service} webhook creation response',
+    })
     return { providerConfigUpdates: { externalId: id } }
   },
 
@@ -342,6 +345,7 @@ export const {service}Handler: WebhookProviderHandler = {
 Trigger outputs use the same schema as block outputs (NOT tool outputs).
 
 **Supported:** `type` + `description` for leaf fields, nested objects for complex data.
+**Also supported:** `nullable: true` and `condition` (`TriggerOutput` in `triggers/types.ts`).
 **NOT supported:** `optional: true`, `items` (those are tool-output-only features).
 
 ```typescript
@@ -377,15 +381,15 @@ apps/sim/lib/webhooks/polling/
 
 ```typescript
 import { pollingIdempotency } from '@/lib/core/idempotency/service'
-import type { PollingProviderHandler, PollWebhookContext } from '@/lib/webhooks/polling/types'
+import type { PollingProviderHandler, PollOutcome, PollWebhookContext } from '@/lib/webhooks/polling/types'
 import { markWebhookFailed, markWebhookSuccess, resolveOAuthCredential, updateWebhookProviderConfig } from '@/lib/webhooks/polling/utils'
 import { processPolledWebhookEvent } from '@/lib/webhooks/processor'
 
 export const {service}PollingHandler: PollingProviderHandler = {
   provider: '{service}',
   label: '{Service}',
 
-  async pollWebhook(ctx: PollWebhookContext): Promise<'success' | 'failure'> {
+  async pollWebhook(ctx: PollWebhookContext): Promise<PollOutcome> {
     const { webhookData, workflowData, requestId, logger } = ctx
     const webhookId = webhookData.id
 

.agents/skills/council/SKILL.md

@@ -2,13 +2,6 @@
 name: council
 description: Spawn parallel task agents to explore a given area of the codebase from multiple angles, then use their findings to answer the question or build a plan. Use when a task needs broad fan-out exploration across many files before acting.
 argument-hint: <area-of-interest>
-# No agents/openai.yaml by design: council is a meta/exploration utility (like cleanup, ship, you-might-not-need-*), not a service-integration builder, so it intentionally ships no standalone agent card.
 ---
 
-Based on the given area of interest, please:
-
-1. Dig around the codebase in terms of that given area of interest, gather general information such as keywords and architecture overview.
-2. Spawn off n=10 (unless specified otherwise) task agents to dig deeper into the codebase in terms of that given area of interest, some of them should be out of the box for variance.
-3. Once the task agents are done, use the information to do what the user wants.
-
-If user is in plan mode, use the information to create the plan.
+Map the area of interest first (keywords, architecture), then fan out parallel agents, each exploring a distinct angle, including a few unconventional ones. Size the fan-out to the area (the user may name a number). Use their findings to answer the question, or to write the plan in plan mode.