Skip to content

Commit c4d0e06

Browse files
committed
fix(dashboard-agent): move the mandatory-sweep imperative into tool descriptions
The system-prompt bullet doesn't bind at decision time; get_run, get_error, get_queue, and correlate_version now carry the imperative in their own descriptions (tools.chars is uncapped). correlate_version also gained project/environment overrides and lost its "dev runs behave this way" claim, which asserted a run's deploy state from a single-environment 404 — the same fabricated-absence bug banned elsewhere. Tool schemas rebuild fresh every turn with no caching or dashboard-override path (unlike the system prompt, which resolves through a managed prompt cached per worker process), so this change is live on the very next call.
1 parent 559fc0b commit c4d0e06

4 files changed

Lines changed: 32 additions & 21 deletions

File tree

internal-packages/dashboard-agent/src/__snapshots__/prompt-prefix.test.ts.snap

Lines changed: 10 additions & 10 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

internal-packages/dashboard-agent/src/tool-api-transport.test.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,7 @@ describe("a broken request reads as a broken request, never as an answer", () =>
136136

137137
const result = await run("correlate_version", { runId: "run_1234" });
138138

139-
expect(result.error).toContain("isn't locked to a deployed version");
139+
expect(result.error).toContain("No commit found for run run_1234 in the current environment");
140+
expect(result.error).not.toContain("isn't locked to a deployed version");
140141
});
141142
});

internal-packages/dashboard-agent/src/tool-api.ts

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -656,23 +656,31 @@ export function buildApiTools(args: {
656656

657657
correlate_version: tool({
658658
...correlateVersionSchema,
659-
execute: async ({ runId }) => {
659+
execute: async ({ runId, project, environment }) => {
660660
if (!hasAuth) return NO_AUTH;
661-
if (!projectRef || !environmentName) {
661+
const effectiveProjectRef = project ?? projectRef;
662+
const effectiveEnvironmentName = environment ?? environmentName;
663+
if (!effectiveProjectRef || !effectiveEnvironmentName) {
662664
return { error: "No current environment is available to resolve the run's version." };
663665
}
666+
const target = crossProjectTarget({ project, environment });
664667
// A user-level route, so this uses the delegated token rather than the env JWT.
668+
// An override drops the branch: it names another project/environment, which
669+
// the current branch can't be assumed to apply to.
665670
const result = await apiGet(
666671
origin,
667-
`/api/v1/projects/${projectRef}/${environmentName}/runs/${encodeURIComponent(runId)}/commit`,
672+
`/api/v1/projects/${effectiveProjectRef}/${effectiveEnvironmentName}/runs/${encodeURIComponent(runId)}/commit`,
668673
userActorToken!,
669-
environmentBranch
674+
target ? undefined : environmentBranch
670675
);
671676
if (!result.ok) {
672-
// Only a real 404 says "no commit"; a transport failure says nothing.
677+
// Only a real 404 says "no commit here"; a transport failure says nothing, and a
678+
// 404 is never evidence the run isn't locked/deployed — only that this environment
679+
// has no record of it. Asserting "dev run" or "no locked commit" from it is the bug.
673680
if ("status" in result && result.status === 404) {
681+
const scope = target ? "that project/environment" : "the current environment";
674682
return {
675-
error: `Run ${runId} isn't locked to a deployed version, so there's no commit to correlate (dev runs behave this way).`,
683+
error: `No commit found for run ${runId} in ${scope}. That is not evidence the run isn't locked to a deployment — sweep (list_projects, then get_run with project/environment) before concluding, then retry this call with project/environment for wherever it's found.`,
676684
};
677685
}
678686
return { error: `Couldn't resolve the commit for ${runId}${fetchReason(result)}.` };

internal-packages/dashboard-agent/src/tool-schemas.ts

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ export const listRunsSchema = tool({
9494

9595
export const getRunSchema = tool({
9696
description:
97-
"Get the status, timing, cost, and error details for a single run in the current environment, by its run id (run_...). The `wait` field is the already-computed queue wait (or, when unreliable, time since creation) — never recompute it from createdAt/startedAt.",
97+
"Get the status, timing, cost, and error details for a single run in the current environment, by its run id (run_...). The `wait` field is the already-computed queue wait (or, when unreliable, time since creation) — never recompute it from createdAt/startedAt. A 404 (in the error message) means this run isn't in the current environment, never that it doesn't exist: you MUST immediately, this same turn, with no permission question, call list_projects and repeat this call with `project`/`environment` set for the current project's other environments and every sibling project's matching environment, then answer naming what you checked.",
9898
inputSchema: z.object({
9999
runId: z.string().describe("The run id, e.g. run_abc123."),
100100
project: projectOverrideField,
@@ -143,7 +143,7 @@ export const listErrorsSchema = tool({
143143

144144
export const getErrorSchema = tool({
145145
description:
146-
"Get the full detail for a single error group by its id (error_...): type, message, occurrence count, first/last seen, affected task versions, and lifecycle state (who resolved/ignored it and when). `recurredSinceResolve` is already computed — true when an occurrence landed after resolvedAt, so never compare those dates yourself. Pair with list_runs(errorId) to see the runs behind it.",
146+
"Get the full detail for a single error group by its id (error_...): type, message, occurrence count, first/last seen, affected task versions, and lifecycle state (who resolved/ignored it and when). `recurredSinceResolve` is already computed — true when an occurrence landed after resolvedAt, so never compare those dates yourself. Pair with list_runs(errorId) to see the runs behind it. A 404 (in the error message) means this error group isn't in the current environment, never that it doesn't exist: you MUST immediately, this same turn, with no permission question, call list_projects and repeat this call with `project`/`environment` set for the current project's other environments and every sibling project's matching environment, then answer naming what you checked.",
147147
inputSchema: z.object({
148148
errorId: z.string().describe("The error group id, e.g. error_abc123, from list_errors."),
149149
project: projectOverrideField,
@@ -211,7 +211,7 @@ export const getReportSchema = tool({
211211

212212
export const getQueueSchema = tool({
213213
description:
214-
"Get one queue's metrics over a window: wait latency (p50/p95), peak depth, how many runs started (throughput), and how often the queue was throttled by its concurrency limit. Use this for 'how deep is the X queue', 'is X backed up', or 'why are runs waiting'. The answer also carries the queue's live row: `paused`, `queuedNow`, `runningNow`, `concurrencyLimit`, and `exists: false` when no queue of that name is there at all. When that read fails rather than answers, `exists` is `\"unknown\"` with a `liveStateError`: the queue's state is unknown, not missing. For a custom queue it also carries `consumerTasks`: the deployed tasks whose queue config names this queue. When present, `slotHolders` (each run's id, status, uri, consistency, phase (`admitted` | `dequeued`), and concurrencyKey) lists the runs holding the queue's concurrency slots, but the list is never guaranteed exhaustive; `slotHolderFacts` (admittedCount, dequeuedCount, runningReported, truncated, unlistedRunning, consistency) is the server-computed snapshot summary — `truncated` or `unlistedRunning > 0` mean there are holders `slotHolders` doesn't list. When present, `concurrency` (current, base, override, overriddenBy, overriddenAt) distinguishes a temporary override from configured `concurrencyLimit`. When present, `envConcurrency` (limit, current, burstFactor) is the environment-wide dequeue gate: the environment saturates at `current >= limit * burstFactor`, not at `current >= limit` (burstFactor defaults to 2, so headroom above the plain limit is often still open) — and `current` is the last-displayed dequeued count, which can lag the number actually gating dequeues. Use these three fields together before naming the environment as the bottleneck; never infer that from throttledCount alone. All are absent on an older API rather than empty. A holder's phase `admitted` (not yet `dequeued`) may legitimately be pending, not a mismatch. Consistency \"mismatch\" on a holder means the scheduler still counts it as a holder though its run state disagrees; on slotHolderFacts it means the scheduler's own counters disagree right now — prefer those facts to comparing runningNow yourself, and never call either \"leaked\" or \"stale\". Consistency `unresolved` means the run id is citable but its state, and slotHolderFacts' counts, are not — don't assert either. Never assert a run is currently executing from runningNow or concurrencyLimit alone, and never say holders are unaccounted for beyond what truncated/unlistedRunning/consistency actually state — 'nothing holds the slots' is never licensed by an incomplete list.",
214+
"Get one queue's metrics over a window: wait latency (p50/p95), peak depth, how many runs started (throughput), and how often the queue was throttled by its concurrency limit. Use this for 'how deep is the X queue', 'is X backed up', or 'why are runs waiting'. The answer also carries the queue's live row: `paused`, `queuedNow`, `runningNow`, `concurrencyLimit`, and `exists: false` when no queue of that name is there at all. When `exists` is `false` in the current environment, you MUST immediately, this same turn, with no permission question, call list_projects and repeat this call with `project`/`environment` set for the current project's other environments and every sibling project's matching environment, then answer naming what you checked. When that read fails rather than answers, `exists` is `\"unknown\"` with a `liveStateError`: the queue's state is unknown, not missing. For a custom queue it also carries `consumerTasks`: the deployed tasks whose queue config names this queue. When present, `slotHolders` (each run's id, status, uri, consistency, phase (`admitted` | `dequeued`), and concurrencyKey) lists the runs holding the queue's concurrency slots, but the list is never guaranteed exhaustive; `slotHolderFacts` (admittedCount, dequeuedCount, runningReported, truncated, unlistedRunning, consistency) is the server-computed snapshot summary — `truncated` or `unlistedRunning > 0` mean there are holders `slotHolders` doesn't list. When present, `concurrency` (current, base, override, overriddenBy, overriddenAt) distinguishes a temporary override from configured `concurrencyLimit`. When present, `envConcurrency` (limit, current, burstFactor) is the environment-wide dequeue gate: the environment saturates at `current >= limit * burstFactor`, not at `current >= limit` (burstFactor defaults to 2, so headroom above the plain limit is often still open) — and `current` is the last-displayed dequeued count, which can lag the number actually gating dequeues. Use these three fields together before naming the environment as the bottleneck; never infer that from throttledCount alone. All are absent on an older API rather than empty. A holder's phase `admitted` (not yet `dequeued`) may legitimately be pending, not a mismatch. Consistency \"mismatch\" on a holder means the scheduler still counts it as a holder though its run state disagrees; on slotHolderFacts it means the scheduler's own counters disagree right now — prefer those facts to comparing runningNow yourself, and never call either \"leaked\" or \"stale\". Consistency `unresolved` means the run id is citable but its state, and slotHolderFacts' counts, are not — don't assert either. Never assert a run is currently executing from runningNow or concurrencyLimit alone, and never say holders are unaccounted for beyond what truncated/unlistedRunning/consistency actually state — 'nothing holds the slots' is never licensed by an incomplete list.",
215215
inputSchema: z.object({
216216
queue: z
217217
.string()
@@ -268,9 +268,11 @@ export const getDeploySchema = tool({
268268

269269
export const correlateVersionSchema = tool({
270270
description:
271-
"Find the exact code a run executed: the deployed version it locked to, that version's commit SHA, and the commit message, branch, and pull request behind it. Use this for 'what commit is this run running', 'which change broke this', or before reading source for a run.",
271+
"Find the exact code a run executed: the deployed version it locked to, that version's commit SHA, and the commit message, branch, and pull request behind it. Use this for 'what commit is this run running', 'which change broke this', or before reading source for a run. A 404 here means not found IN THIS environment, never that the run isn't locked or deployed: you MUST immediately, this same turn, with no permission question, call list_projects and repeat this call with `project`/`environment` set for the current project's other environments and every sibling project's matching environment, then answer naming what you checked. Never infer 'dev run' or 'no locked commit' from a single-environment 404.",
272272
inputSchema: z.object({
273273
runId: z.string().describe("The run id, e.g. run_abc123."),
274+
project: projectOverrideField,
275+
environment: environmentOverrideField,
274276
}),
275277
});
276278

0 commit comments

Comments
 (0)