Skip to content

feat(claude-agent-sdk): capture gen_ai.skill.* on Skill load execute_tool span#226

Open
sipercai wants to merge 1 commit into
alibaba:mainfrom
sipercai:feat/claude-agent-sdk-skill-attrs
Open

feat(claude-agent-sdk): capture gen_ai.skill.* on Skill load execute_tool span#226
sipercai wants to merge 1 commit into
alibaba:mainfrom
sipercai:feat/claude-agent-sdk-skill-attrs

Conversation

@sipercai

Copy link
Copy Markdown
Collaborator

What

Capture gen_ai.skill.{name,id,description,version} on the execute_tool span of the Claude Agent SDK built-in Skill tool.

Telemetry is bound to the ToolUseBlock(name="Skill") tool span (not the subsequent UserMessage TextBlock that injects SKILL.md content).

Attribute mapping (per spec)

Source Span attribute
ToolUseBlock.input.skill (frontmatter name fallback) gen_ai.skill.name
claude:project:<skill-name> gen_ai.skill.id
SKILL.md frontmatter description gen_ai.skill.description
SKILL.md frontmatter version gen_ai.skill.version
  • cwd is captured from SystemMessage.data.cwd to locate the project-level .claude/skills/<name>/SKILL.md.
  • If start info is incomplete, skill.name/skill.id fall back to UserMessage.tool_use_result.commandName before the span closes.
  • All metadata reads are best-effort: failures are swallowed and never propagate to the SDK call.

Local validation

  • ruff check instrumentation-loongsuite/loongsuite-instrumentation-claude-agent-sdk → all checks passed
  • pytest test_span_validation.py::test_skill_tool_span_attributes → 1 passed (exactly one gen_ai.tool.name=Skill execute_tool span with all four gen_ai.skill.* attributes, skill.id = claude:project:probe-skill)
  • pytest test_span_validation.py::test_skill_metadata_read_failure_does_not_break_sdk → 1 passed (best-effort: missing SKILL.md does not break the call)
  • Regression (test_span_validation.py + test_with_cassettes.py + test_task_subagent_real_data.py) → 24 passed, no regressions

Draft PR; not requesting review yet.

@CLAassistant

CLAassistant commented Jun 22, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

…tool span

Attach gen_ai.skill.name/id/description/version to the execute_tool span of
the built-in Skill tool. Telemetry is bound to the ToolUseBlock(name="Skill")
tool span (not the SKILL.md-injecting UserMessage TextBlock).

- skill.name from ToolUseBlock.input.skill (frontmatter.name fallback)
- skill.id = claude:project:<skill-name>
- skill.description/version read best-effort from <cwd>/.claude/skills/<name>/SKILL.md
  frontmatter (cwd from SystemMessage.data.cwd)
- fallback to UserMessage.tool_use_result.commandName when start info incomplete
- metadata read failures never propagate to the SDK call

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sipercai sipercai force-pushed the feat/claude-agent-sdk-skill-attrs branch from b1a7531 to 7f78738 Compare June 22, 2026 02:53
@sipercai sipercai marked this pull request as ready for review June 22, 2026 03:07

@ralf0131 ralf0131 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Captures gen_ai.skill.{name,id,description,version} on the Skill tool execute_tool span. The cwd propagation chain and best-effort fallback design are solid. Test coverage is good (happy path + failure case). However, one critical issue must be fixed before merge: the top-level import yaml will break the entire instrumentation module if PyYAML is not installed.

Findings

  • [Critical] patch.py:22 — import yaml at module level without pyyaml in pyproject.toml dependencies; will crash the entire module on import if PyYAML is missing
  • [Info] patch.py:98 — suggestion for lazy-import placement if that approach is chosen

Suggestions

Move import yaml inside _parse_skill_frontmatter (the only consumer) to preserve the best-effort guarantee, or add pyyaml to pyproject.toml dependencies.

Note: the PR body mentions "Draft PR; not requesting review yet" — consider converting to a GitHub Draft PR to prevent accidental merge before this is addressed.


Automated review by github-manager-bot

import time
from typing import Any, Dict, List, Optional

import yaml

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] import yaml at module level without pyyaml in pyproject.toml dependencies. If PyYAML is not installed in the user's environment, the entire patch.py will fail to import with ModuleNotFoundError, breaking the instrumentation completely — not just the skill metadata feature. This contradicts the best-effort design intent. Fix: either add pyyaml to dependencies in pyproject.toml, or move the import inside _parse_skill_frontmatter (lazy import) so only the skill parsing is affected when PyYAML is missing.

# skill id prefix for project-scoped Claude Agent SDK skills.
_SKILL_ID_PREFIX = "claude:project:"


Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Info] If you choose the lazy-import approach, the yaml.safe_load(frontmatter_text) call here is the only place yaml is used — adding import yaml at the top of this function's try block would keep the scope minimal and preserve the best-effort guarantee.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants