⚡ Claude Token Optimization2026-07-18 — Red-Team Benchmark

[github-actions]

Target Workflow: Red-Team Benchmark

Source report: No token-usage-report issue found (gh not authenticated in sandbox); analysis based on /tmp/gh-aw/token-audit/claude-logs.json
Estimated cost per run: ~$0.010
Total tokens per run: ~7,894
Cache read rate: N/A (haiku-4-5, data not available in usage artifact)
Cache write rate: N/A
LLM turns: ≤8 (max-turns: 8; likely 2–3 actual)

Current Configuration

Setting	Value
Model	claude-haiku-4-5
Tools loaded	bash, github[issues]
Tools actually used	bash, safeoutputs (github tools likely unused)
Network groups	github
Pre-agent steps	✅ Yes — 8 steps
Prompt size	~2,750 chars (~687 tokens)

Recommendations

1. Remove github: toolset — it is never called

The agent prompt explicitly instructs the agent to use safeoutputs tools (create_issue or noop), not GitHub MCP tools. The github: toolsets: [issues] block loads ~3 tools (~1,500–2,000 tokens per turn) that are dead weight.

Estimated savings: ~1,500–2,000 tokens/turn × 2–3 turns = ~3,000–6,000 tokens/run (~40–75%)

Implementation — remove from tools: section:

# Before
tools:
  bash: true
  github:
    toolsets: [issues]

# After
tools:
  bash: true

Also remove github from network.allowed: since the agent no longer needs to call the GitHub API:

# Before
network:
  allowed:
    - github

# After
network:
  allowed: []

⚠️ Verify the agent doesn't use any GitHub read tool (e.g., to look up the report issue number). The current prompt references a {report-number} but the benchmark is self-contained in local files, so no GitHub reads are needed.

---

2. Reduce max-turns from 8 to 3

The agent task is simple: (1) run a fixed bash command, (2) read JSON output, (3) call safeoutputs. It requires at most 3 turns. max-turns: 8 allows unnecessary extra turns if the agent loops.

Estimated savings: Prevents runaway loops; ~0 tokens saved in the happy path but eliminates worst-case 5× overspend.

Implementation:

# Before
max-turns: 8

# After
max-turns: 3

---

3. Simplify the prompt — move bash command inline, remove redundant context

The prompt body (2,750 chars) contains:

• A verbose 5-command bash block that duplicates what the pre-agent steps already computed
• A repeated "Issue Format" section that could be a brief template

The agent only needs to know: (a) which files to read, (b) what issue to create. Strip the ## Benchmark Overview and ## Issue Format detail into shorter instructions.

Estimated savings: ~150–200 tokens/turn (prompt reduction from ~687 → ~500 tokens)

Example condensed prompt:

Read `/tmp/gh-aw/agent/benchmark-summary.json`, `/tmp/gh-aw/agent/baseline/summary.json`, and `/tmp/gh-aw/agent/awf/summary.json` (use `{"skipped":true}` if missing). Also `grep -c DENIED /tmp/gh-aw/agent/squid-access.log` for blocked count.

If all runs were skipped (missing API keys), call `noop`. Otherwise create an issue titled `AWF Red-Team Benchmark — YYYY-MM-DD — PASS/FAIL` with a metrics table (run_id, timestamp, scenarios, baseline leaks, AWF leaks, blocked requests, effective ✅/❌), top blocked domains, and a one-paragraph assessment.

---

Cache Analysis

No per-turn cache breakdown is available in the usage artifact (only total token_usage: 7894 is recorded). To get Anthropic-specific cache write/read splits, enable detailed token logging or download the full agent artifact (--artifacts agent).

With claude-haiku-4-5:

• Cache write: ~$0.30/M tokens (haiku)
• Cache read: ~$0.03/M tokens (haiku)
• At ~7,894 tokens/run, cache savings are modest (<$0.001/run)

---

Expected Impact

Metric	Current	Projected	Savings
Total tokens/run	~7,894	~2,500–4,000	~50–70%
Cost/run	~$0.010	~$0.003–0.005	~50–70%
LLM turns	≤8	≤3	−5 max
GitHub API calls	10	~0	−10

---

Implementation Checklist

• Remove github: from tools: in .github/workflows/red-team-benchmark.md
• Remove github from network.allowed: (if agent doesn't need GitHub reads)
• Change max-turns: 8 → max-turns: 3
• Condense prompt body to ~500 tokens (remove Benchmark Overview, shorten Issue Format)
• Recompile: gh aw compile .github/workflows/red-team-benchmark.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Verify CI passes on PR
• Compare token usage on new run vs baseline (~7,894 tokens)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Daily Claude Token Optimization Advisor · 43.7 AIC · ⊞ 6.3K · ◷