Skip to content

feat(accounting): report deduplicated SDK result token usage - #954

Open
ran411285752 wants to merge 5 commits into
agentclientprotocol:mainfrom
ran411285752:fix/token-accounting-usage
Open

feat(accounting): report deduplicated SDK result token usage#954
ran411285752 wants to merge 5 commits into
agentclientprotocol:mainfrom
ran411285752:fix/token-accounting-usage

Conversation

@ran411285752

Copy link
Copy Markdown
Contributor

Purpose

Preserve provider token accounting for failed, cancelled-result, autonomous, background, and subagent work without changing existing ACP turn-usage semantics.

Closes #953.

Related to #927 and #914.

Protocol impact

  • Adds a negotiated _claude/accountingUsage version 1 extension notification.
  • Advertises support through agentCapabilities._meta.claudeCode.accountingUsage.
  • Requires the Host to opt in with clientCapabilities._meta.claudeCode.accountingUsage.version = 1.
  • Does not change standard usage_update.
  • Does not change PromptResponse.usage.

Accounting semantics

  • resultId is the stable deduplication key.
  • Only params.usage is additive.
  • modelUsage is a cumulative SDK audit snapshot and must not be summed directly.
  • resultUsage is diagnostic top-level result usage and must not be added to params.usage.
  • PromptResponse.usage and accounting events must not be summed together.
  • Consecutive aggregate model snapshots are differenced after aggregating all model keys.
  • Invalid snapshots use an explicit fallback source.
  • Fallback usage is reconciled when cumulative snapshots resume, avoiding double counting.
  • Snapshot resets start a new accounting epoch.

Lifecycle coverage

Accounting is attempted before:

  • failed result rejection;
  • cancellation exits;
  • autonomous result exits;
  • normal result settlement.

Optional extension delivery failures are logged without changing the standard result lifecycle.

Evidence

Built-Adapter ACP subprocess scenarios covered:

  • ordinary multi-turn usage;
  • same-model and cross-model subagents;
  • background and autonomous results;
  • error results;
  • cancellation after non-zero assistant usage;
  • resume;
  • compaction;
  • model switching.

A-H and J:

ACP accounting:             304,617
Unique public model ledger: 304,617
Difference:                       0

Compaction scenario I:

Final cumulative modelUsage: 194,914
ACP accounting sum:          194,914
Difference:                        0

The /compact result had zero resultUsage and a 32,903-token accounting delta. Public transcript APIs and JSONL omit that compact-call usage, so they cannot be used as a complete post-compaction accounting ledger.

Verification

  • npm run build
  • npm run lint
  • Prettier check for all five changed files
  • 78 focused accounting/lifecycle tests
  • Full suite: 711 passed, 20 skipped
  • Six pre-existing Windows path-separator assertions remain failing, matching clean origin/main
  • Mandatory review: no Critical, High, or Medium findings

Related context

Why:
- SDK result modelUsage is cumulative and cannot be summed directly.

What:
- Normalize provider token buckets, aggregate model rows, and compute result deltas.
- Deduplicate result UUIDs and expose reset/fallback semantics.

Impact:
- No ACP wire behavior changes until the negotiated integration is enabled.
Why:
- PromptResponse cannot carry usage for failed or autonomous SDK results.

What:
- Advertise a versioned Claude accounting extension.
- Emit one deduplicated accounting notification before result lifecycle exits.

Impact:
- Standard PromptResponse and usage_update behavior remains unchanged.
- Hosts must opt in through client capabilities.
Why:
- Hosts need an explicit additive contract that avoids double counting standard turn usage.
- Review exposed fallback reconciliation and extension transport edge cases.

What:
- Document version negotiation, additive fields, deduplication, and compaction limits.
- Reconcile fallback usage with later cumulative snapshots and reject partial snapshots.
- Keep standard result handling alive when optional accounting delivery fails.
- Add lifecycle and fallback-debt regression coverage.

Impact:
- PromptResponse.usage and standard usage_update semantics remain unchanged.
- Local evidence and raw session data remain uncommitted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add deduplicated token accounting for failed and autonomous SDK results

1 participant