feat(claude-agent-sdk): capture gen_ai.skill.* on Skill load execute_tool span#226
feat(claude-agent-sdk): capture gen_ai.skill.* on Skill load execute_tool span#226sipercai wants to merge 1 commit into
Conversation
…tool span Attach gen_ai.skill.name/id/description/version to the execute_tool span of the built-in Skill tool. Telemetry is bound to the ToolUseBlock(name="Skill") tool span (not the SKILL.md-injecting UserMessage TextBlock). - skill.name from ToolUseBlock.input.skill (frontmatter.name fallback) - skill.id = claude:project:<skill-name> - skill.description/version read best-effort from <cwd>/.claude/skills/<name>/SKILL.md frontmatter (cwd from SystemMessage.data.cwd) - fallback to UserMessage.tool_use_result.commandName when start info incomplete - metadata read failures never propagate to the SDK call Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
b1a7531 to
7f78738
Compare
ralf0131
left a comment
There was a problem hiding this comment.
Summary
Captures gen_ai.skill.{name,id,description,version} on the Skill tool execute_tool span. The cwd propagation chain and best-effort fallback design are solid. Test coverage is good (happy path + failure case). However, one critical issue must be fixed before merge: the top-level import yaml will break the entire instrumentation module if PyYAML is not installed.
Findings
- [Critical] patch.py:22 —
import yamlat module level withoutpyyamlinpyproject.tomldependencies; will crash the entire module on import if PyYAML is missing - [Info] patch.py:98 — suggestion for lazy-import placement if that approach is chosen
Suggestions
Move import yaml inside _parse_skill_frontmatter (the only consumer) to preserve the best-effort guarantee, or add pyyaml to pyproject.toml dependencies.
Note: the PR body mentions "Draft PR; not requesting review yet" — consider converting to a GitHub Draft PR to prevent accidental merge before this is addressed.
Automated review by github-manager-bot
| import time | ||
| from typing import Any, Dict, List, Optional | ||
|
|
||
| import yaml |
There was a problem hiding this comment.
[Critical] import yaml at module level without pyyaml in pyproject.toml dependencies. If PyYAML is not installed in the user's environment, the entire patch.py will fail to import with ModuleNotFoundError, breaking the instrumentation completely — not just the skill metadata feature. This contradicts the best-effort design intent. Fix: either add pyyaml to dependencies in pyproject.toml, or move the import inside _parse_skill_frontmatter (lazy import) so only the skill parsing is affected when PyYAML is missing.
| # skill id prefix for project-scoped Claude Agent SDK skills. | ||
| _SKILL_ID_PREFIX = "claude:project:" | ||
|
|
||
|
|
There was a problem hiding this comment.
[Info] If you choose the lazy-import approach, the yaml.safe_load(frontmatter_text) call here is the only place yaml is used — adding import yaml at the top of this function's try block would keep the scope minimal and preserve the best-effort guarantee.
What
Capture
gen_ai.skill.{name,id,description,version}on theexecute_toolspan of the Claude Agent SDK built-inSkilltool.Telemetry is bound to the
ToolUseBlock(name="Skill")tool span (not the subsequentUserMessageTextBlockthat injectsSKILL.mdcontent).Attribute mapping (per spec)
ToolUseBlock.input.skill(frontmatternamefallback)gen_ai.skill.nameclaude:project:<skill-name>gen_ai.skill.idSKILL.mdfrontmatterdescriptiongen_ai.skill.descriptionSKILL.mdfrontmatterversiongen_ai.skill.versioncwdis captured fromSystemMessage.data.cwdto locate the project-level.claude/skills/<name>/SKILL.md.skill.name/skill.idfall back toUserMessage.tool_use_result.commandNamebefore the span closes.Local validation
ruff check instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk→ all checks passedpytest test_span_validation.py::test_skill_tool_span_attributes→ 1 passed (exactly onegen_ai.tool.name=Skillexecute_toolspan with all fourgen_ai.skill.*attributes,skill.id = claude:project:probe-skill)pytest test_span_validation.py::test_skill_metadata_read_failure_does_not_break_sdk→ 1 passed (best-effort: missingSKILL.mddoes not break the call)test_span_validation.py+test_with_cassettes.py+test_task_subagent_real_data.py) → 24 passed, no regressionsDraft PR; not requesting review yet.