[audit-workflows] Daily Agentic Workflow Audit — 2026-05-14

[github-actions[bot]]

Summary

Clean run, zero failures. Across 79 workflow runs in the last 24 hours, 14 were real agent runs (8 succeeded, 6 still in progress at audit time, 0 failed). The remaining 65 were Dependabot-triggered smoke workflows that correctly skipped on label/policy gates — the high error count (86) is downstream noise from those skips, not real failures.

• Total runs: 79 · Agent runs: 14 · Real failures: 0
• Effective tokens: 52.1M · Estimated cost: $6.53 · Action minutes: 138
• Engine split (agent runs): copilot 10, claude 4
• Firewall: 174/174 requests allowed (0 blocked) across 4 approved domains
• Missing tools: 0 · MCP failures: 0

Key Findings

1. No real failures. Every agent run that reached the engine either succeeded or is still running.
2. 65 cancelled runs are false-positive noise. Smoke workflows triggered by a Dependabot PR (dependencies label) evaluate pre_activation.if to false (gates expect water or other specific labels), correctly skip the agent step, then the conclusion job tries to download an agent artifact that was never produced — producing ##[error]Unable to download artifact(s): Artifact not found for name: agent. This inflates error_count per skipped run without representing a real failure.
3. Heaviest token spender: Daily Safe Output Tool Optimizer (claude, 13.1 min, ~65 outbound requests to api.anthropic.com).
4. Cross-run anomaly: 1 medium-severity insight flagged 4 events with anomaly score > 0.6 (new log template, rare cluster). Worth watching on the next audit.
5. First audit cycle. Repo memory was empty — today's snapshot is now seeded so future audits can compute multi-day trends.

Workflow health (agent runs only, per hour). All recorded agent runs in this window were either successful or still in progress; no failures occurred, so the success-rate line tracks at 100% wherever data exists. The 30-day window currently contains only one day of data — multi-day trends will materialize over subsequent audits.

Token usage (bars, effective tokens) and estimated cost (line) per hour. A pronounced spike around 18–19 UTC corresponds to the Daily Safe Output Tool Optimizer run (~21M effective tokens, the single largest contributor to today's $6.53 spend).

Engine Usage

Engine	Agent Runs
copilot	10
claude	4

Successful Agent Workflows (8)

Workflow	Engine	Duration	Action Min
Smoke CI (x2)	copilot	2.9m / 3.0m	3 / 4
Contribution Check	copilot	5.8m	6
Daily Project Performance Summary Generator	copilot	8.3m	9
Daily Caveman Optimizer	claude	8.7m	9
PR Sous Chef	copilot	7.3m	8
Copilot PR Prompt Pattern Analysis	copilot	6.0m	6
Daily Safe Output Tool Optimizer	claude	13.1m	14

In-Progress Agent Workflows (6)

• Test Quality Sentinel (copilot)
• Daily Regulatory Report Generator (copilot)
• Agentic Workflow Audit Agent (claude — this run)
• PR Sous Chef (copilot)
• Design Decision Gate (claude)
• Matt Pocock Skills Reviewer (copilot)

Filtered-Skip Pattern (65 runs)

All 65 cancelled runs share the same shape:

• Triggered by Dependabot pull_request (action=labeled, label=dependencies)
• pre_activation.if expression resolves to false (label gate expects water etc.)
• Agent step skips with success() == false
• Conclusion job emits ##[error]Unable to download artifact(s): Artifact not found for name: agent

Affected workflows (smoke suite): Smoke Multi PR, Smoke OTEL, Smoke OpenCode, Smoke Pi, Smoke Temporary ID, Smoke Copilot ARM64, Smoke Claude, Smoke Codex, Smoke Gemini, Smoke Project, Smoke Agent (all variants: scoped/approved, public/approved, public/none, all/merged, all/none), Smoke Create Cross-Repo PR, Smoke Update Cross-Repo PR, Smoke Call Workflow, Smoke Crush, Code Refiner, Changeset Generator, Design Decision Gate, Agent Container Smoke Test.

Recommendation: guard the artifact-download step in the conclusion job on the agent step's outcome != 'skipped', or downgrade missing-artifact-on-skip to a warning, so the audit signal isn't dominated by expected skips.

Firewall Traffic by Workflow

Workflow	Requests	Domains
Daily Safe Output Tool Optimizer	65	api.anthropic.com
Contribution Check	27	api.githubcopilot.com
Daily Project Performance Summary Generator	26	api.githubcopilot.com, pypi.org, files.pythonhosted.org
Daily Caveman Optimizer	18	api.anthropic.com
Copilot PR Prompt Pattern Analysis	11	api.githubcopilot.com
PR Sous Chef	10	api.githubcopilot.com
Smoke CI	2	api.githubcopilot.com

All 174 requests allowed; nothing blocked.

Observability Insights

• [info] Actuation mix: Across 79 runs, 0 executed write-capable safe outputs; all stayed read-only.
• [info] Log template patterns: Mined 89 distinct event templates across 6 pipeline stages from 92 events in 79 runs.
• [medium] Cross-run anomaly: 4 events flagged with anomaly score > 0.6 (new log templates discovered, rare cluster).

Recommendations

1. Reduce false-positive errors from skipped smoke workflows — guard conclusion-job artifact download on agent-step outcome, or downgrade missing-artifact-on-skip to a warning. (Currently inflating total_errors from ~0 real to 86 reported.)
2. Re-investigate the 4 anomalous log templates on the next audit — if the same anomalies persist, examine whether they indicate a real behavioral drift in a specific workflow.
3. Continue daily audits — the trend history file (metrics/daily.jsonl) is now seeded; in a few cycles the 30-day trend charts will become genuinely informative.

References

• §25887034189 — this audit run
• §25886035116 — Daily Safe Output Tool Optimizer (heaviest token user)
• §25885453198 — Daily Caveman Optimizer (successful claude run)

Generated by 🔍 Agentic Workflow Audit Agent · ● 18.9M · ◷

• expires on May 15, 2026, 9:50 PM UTC

[pelikhan]

/q fix repo memory. There should be more memories.

[github-actions[bot]]

🔧 Pay attention, 007! Q is preparing your gadgets for this discussion comment...

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-05-15T21:50:04.316Z.

Closed by Workflow