feat(node): Add Claude Code Agent SDK instrumentation #17844
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![seer-by-sentry[bot]](https://avatars.githubusercontent.com/in/801464?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
size-limit report 📦
|
node-overhead report 🧳Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.
|
codyde
changed the title
codyde
changed the title
|
|
||
| type ClaudeCodeInstrumentationOptions = ClaudeCodeOptions; | ||
|
|
||
| const GEN_AI_ATTRIBUTES = { |
There was a problem hiding this comment.
We already have these attributes in packages/core/src/utils/ai/gen-ai-attributes.ts
|
|
||
| const SENTRY_ORIGIN = 'auto.ai.claude-code'; | ||
|
|
||
| function setTokenUsageAttributes( |
There was a problem hiding this comment.
can you reuse function from packages/core/src/utils/ai/utils.ts?
| // Parse query arguments | ||
| const [queryParams] = args as [Record<string, unknown>]; | ||
| const { options: queryOptions, inputMessages } = queryParams || {}; | ||
| const model = (queryOptions as Record<string, unknown>)?.model ?? 'sonnet'; |
There was a problem hiding this comment.
can we revert to unknown if not model found here? i think this might be confusing if it's not accurate
| name: CLAUDE_CODE_INTEGRATION_NAME, | ||
| options, | ||
| setupOnce() { | ||
| // Note: Automatic patching via require hooks doesn't work for ESM modules |
There was a problem hiding this comment.
actually i believe InstrumentationModuleDefinition will automatically patch Node.js modules when they're loaded via import, you can find some patterns in other AI integrations e.g anthropic AI
| * Patches the Claude Code SDK query function with Sentry instrumentation. | ||
| * This function can be called directly to patch an imported query function. | ||
| */ | ||
| export function patchClaudeCodeQuery( |
There was a problem hiding this comment.
We should hook into the query and use proxy here
| ); | ||
|
|
||
| // Preserve Query interface methods | ||
| if (typeof (originalQueryInstance as Record<string, unknown>).interrupt === 'function') { |
There was a problem hiding this comment.
using proxy should clean this up a little
Thanks SO much for all these. I'll get started on them. I tried REALLY hard to figure out how to hook into the existing query, and I couldn't get it to work no matter what I tried. I'll chat with you in slack on it, but I'd love some advice / guidance. I tried a bunch of different angles - but each time I ran into effectively timing issues where we couldn't hook fast enough. Felt like a limitation on how Claude Code's SDK works - but could be a total skill issue on my side. |
Hello @codyde, are you still working on this? if not, let's close this, we're trying to clean up the stale PRs |
I definitely am! I pushed up a few more commits today that included fixes for some of the other items you mentioned - but im struggling to get through this proxy one. I might need some pairing time to take a look at it together since im less familiar with the functionality. |
codyde
force-pushed
the
claude-code-agent-instrumentation
branch
from
4df75cc to
6e267e0
Compare
Adds Sentry tracing instrumentation for the @anthropic-ai/claude-agent-sdk
following OpenTelemetry Semantic Conventions for Generative AI.
Key features:
- Captures agent invocation, LLM chat, and tool execution spans
- Records token usage, model info, and session tracking
- Supports input/output recording based on sendDefaultPii setting
- Provides createInstrumentedClaudeQuery() helper for clean DX
Due to ESM-only module constraints, this integration uses a helper function
pattern instead of automatic OpenTelemetry instrumentation hooks.
Usage:
```typescript
import { createInstrumentedClaudeQuery } from '@sentry/node';
const query = createInstrumentedClaudeQuery();
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
…g in Claude Code integration - Add SEMANTIC_ATTRIBUTE_SENTRY_OP to all span creation calls (invoke_agent, chat, execute_tool) - Capture exceptions to Sentry in catch block with proper mechanism metadata - Ensure child spans (currentLLMSpan, previousLLMSpan) are always closed in finally block - Prevents incomplete traces if generator exits early
…umentation
- Add OpenTelemetry-based automatic instrumentation via SentryClaudeCodeAgentSdkInstrumentation
- Extract ClaudeCodeOptions to dedicated types.ts file
- Remove backwards compatibility exports (patchClaudeCodeQuery, createInstrumentedClaudeQuery)
- Rename integration to claudeCodeAgentSdkIntegration
- Register instrumentation in OTEL preload for automatic patching
- Update NextJS re-exports to match simplified API
Users now only need:
```typescript
Sentry.init({ integrations: [Sentry.claudeCodeAgentSdkIntegration()] });
import { query } from '@anthropic-ai/claude-agent-sdk'; // Auto-instrumented
```
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>
…ec compliance - Fix GEN_AI_SYSTEM_ATTRIBUTE to use 'anthropic' per OpenTelemetry semantic conventions - Add GEN_AI_REQUEST_AVAILABLE_TOOLS_ATTRIBUTE for capturing available tools from system init - Add GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE for tracking stop_reason - Use getTruncatedJsonString for proper payload truncation in span attributes - Expand tool categorization with new tools (KillBash, EnterPlanMode, AskUserQuestion, Skill, MCP tools) - Add better error metadata with function name in mechanism data - Export patchClaudeCodeQuery for manual instrumentation use cases - Add comprehensive integration tests for Claude Code Agent SDK instrumentation
codyde
force-pushed
the
claude-code-agent-instrumentation
branch
from
6e267e0 to
3f69bfd
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| async *_createGenerator(params) { | ||
| const model = params.options?.model || 'claude-sonnet-4-20250514'; | ||
| const sessionId = `sess_${Math.random().toString(36).substr(2, 9)}`; |
Check failure
Code scanning / CodeQL
Insecure randomness High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 1 day ago
General fix: Replace use of Math.random() for generating identifiers with a cryptographically secure generator, such as crypto.randomBytes in Node.js. Convert securely generated bytes into a URL-safe or base36-like string for use in the sessionId, keeping the format and length similar to the original where practical.
Best concrete fix here: In _createGenerator, change the sessionId definition on line 43 to use crypto.randomBytes(6).toString('base64url') (or similar) instead of Math.random().... Add an import of Node’s built-in crypto module at the top of this file. This keeps the rest of the logic intact: sessionId is still a short string with an alphanumeric-ish prefix sess_, just the random part is generated securely.
Specific changes:
- In
dev-packages/node-integration-tests/suites/tracing/claude-code/mock-server.mjs, addimport crypto from 'crypto';at the top. - Replace line 43’s
sessionIdassignment:- From:
const sessionId = \sess_${Math.random().toString(36).substr(2, 9)}`;` - To:
const sessionId = \sess_${crypto.randomBytes(6).toString('base64url')}`;`
- From:
- No other code paths need adjustment because all uses of
sessionIdtreat it as an opaque string.
| @@ -1,4 +1,5 @@ | ||
| /* eslint-disable no-console, max-lines */ | ||
| import crypto from 'crypto'; | ||
| /** | ||
| * Mock implementation of @anthropic-ai/claude-agent-sdk | ||
| * Simulates the query function behavior for testing | ||
| @@ -40,7 +41,7 @@ | ||
|
|
||
| async *_createGenerator(params) { | ||
| const model = params.options?.model || 'claude-sonnet-4-20250514'; | ||
| const sessionId = `sess_${Math.random().toString(36).substr(2, 9)}`; | ||
| const sessionId = `sess_${crypto.randomBytes(6).toString('base64url')}`; | ||
| const scenarioName = params.options?.scenario || 'basic'; | ||
|
|
||
| // Get scenario or use default |
codyde
changed the title
|
|
||
| export type ClaudeCodeInstrumentationOptions = ClaudeCodeOptions; | ||
|
|
||
| const SENTRY_ORIGIN = 'auto.ai.claude-code'; |
There was a problem hiding this comment.
Invalid hyphen in span origin violates trace specification (Bugbot Rules)
The SENTRY_ORIGIN constant is set to 'auto.ai.claude-code' which contains a hyphen character. According to the rules file, a proper origin must only contain [a-z], [A-Z], [0-9], _ and . characters. The hyphen violates this specification. This value is used for both the SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN span attribute and the mechanism.type in captureException. Looking at other AI integrations, the correct format would be 'auto.ai.claude_code' using an underscore instead. See the trace origin specification: https://develop.sentry.dev/sdk/telemetry/traces/trace-origin/
Additional Locations (1)
| const query1 = mockSdk.query({ | ||
| prompt: 'Read the file', | ||
| options: { model: 'claude-sonnet-4-20250514', scenario: 'withTools' }, | ||
| }); |
There was a problem hiding this comment.
Test scenarios missing instrumentation patching step (Bugbot Rules)
The test scenarios scenario-tools.mjs and scenario-errors.mjs call mockSdk.query() directly without patching it with patchClaudeCodeQuery. The working scenarios (scenario.mjs and scenario-simple.mjs) properly call patchClaudeCodeQuery(originalQuery, ...) before using the query function. Without patching, no instrumentation runs, so no spans or error events are created. The tests in test.ts that rely on these scenarios (tool execution tests and error handling tests) won't actually test the instrumentation's behavior. This violates the review rule: "Check that tests actually test the newly added behaviour."
Summary
Adds Sentry tracing instrumentation for the
@anthropic-ai/claude-agent-sdk(Claude Code Agent SDK) following OpenTelemetry Semantic Conventions for Generative AI.This integration enables AI monitoring for Claude Code agents with comprehensive telemetry:
gen_ai.invoke_agent)gen_ai.chat)gen_ai.execute_tool)sendDefaultPii)Implementation
Uses automatic OpenTelemetry instrumentation via
import-in-the-middlehooks - the same pattern as other AI integrations (Anthropic, OpenAI, Vercel AI, etc.). When the integration is added, it automatically patches thequeryfunction from@anthropic-ai/claude-agent-sdk.Important: Sentry must be initialized before importing
@anthropic-ai/claude-agent-sdkfor auto-instrumentation to work.Usage
Options
recordInputsbooleansendDefaultPiirecordOutputsbooleansendDefaultPiiagentNamestring'claude-code'Captured Telemetry
Span Hierarchy
Attributes (OpenTelemetry GenAI Semantic Conventions)
gen_ai.system:anthropicgen_ai.operation.name:invoke_agent|chat|execute_toolgen_ai.agent.name: Custom orclaude-codegen_ai.request.model: Model identifiergen_ai.request.available_tools: Available tools from system initgen_ai.response.id: Response/session IDgen_ai.response.model: Actual model usedgen_ai.response.finish_reasons: Stop reasongen_ai.response.text: Response text (whenrecordOutputs: true)gen_ai.response.tool_calls: Tool calls made (whenrecordOutputs: true)gen_ai.tool.name: Tool name (e.g.,Read,Bash,WebSearch)gen_ai.tool.type:function|extension|datastoregen_ai.tool.input: Tool input (whenrecordInputs: true)gen_ai.tool.output: Tool output (whenrecordOutputs: true)gen_ai.usage.input_tokens: Input token countgen_ai.usage.output_tokens: Output token countgen_ai.usage.cache_creation_input_tokens: Cache creation tokensgen_ai.usage.cache_read_input_tokens: Cache read tokensTool Type Classification
Bash,Read,Write,Edit,Glob,Grep,Task,TodoWrite,NotebookEdit,SlashCommand,AskUserQuestion,Skill, etc.WebSearch,WebFetch,ListMcpResources,ReadMcpResourceManual Instrumentation
For advanced use cases where auto-instrumentation is not suitable,
patchClaudeCodeQueryis exported for manual patching:Testing
Includes comprehensive integration tests covering:
sendDefaultPii: truerecordInputs/recordOutputsoptions