Skip to content

feat: modularize oracle, query-opt, and replication skills#2

Open
looreenz wants to merge 1 commit into
MariaDB:mainfrom
looreenz:feat/skill-modularization-part2
Open

feat: modularize oracle, query-opt, and replication skills#2
looreenz wants to merge 1 commit into
MariaDB:mainfrom
looreenz:feat/skill-modularization-part2

Conversation

@looreenz

Copy link
Copy Markdown

Continued implementation of the modular refactoring, applying the new architecture to high-density technical skills.

Key Changes:

  • Atomic Content Migration: Decomposed monolithic reference content from oracle-to-mariadb, mariadb-query-optimization, and mariadb-replication-and-ha into topic-specific files within docs/.
  • Deep Dive Indexing: Root SKILL.md files now feature a compact navigation index linking directly to specific sub-topics (e.g., Execution Plans, Galera Cluster, PL/SQL Syntax).
  • Schema Consistency: Ensured standard formatting for "LLM Hallucination" tables and "Best Practices" lists across all three skills, facilitating consistent parsing by AI agents.

@mariadb-stefan-hinz

Copy link
Copy Markdown

@looreenz You suggested to split up those fairly large (100-200 lines) Skill files into modular ones, with a skill-registry.md file as the index. The modular files are just are just 10-20 lines. I asked Claude if such a granular split make sense in terms of saving tokens and performance.

Short answer: no — as proposed, it's a net loss on both tokens and latency, not a win. The PR author's "lazy-loading" framing rests on a misunderstanding of how Anthropic Skills actually load.

Three reasons the split is a pessimization

  1. Skills load as a single unit. When a skill activates, the agent reads the entire SKILL.md body in one shot. There is no machinery that automatically loads "only the relevant subsection." If the file is split into 13 atomic docs, the agent's SKILL.md is now an index, and the agent has to make a deliberate Read tool call for each atomic file it wants. That's a multi-turn round-trip per file — system framing, tool-use blocks, results — and the per-call overhead easily exceeds the tokens "saved" by skipping unrelated sections.
  2. Trap tables need their cross-references. Robert's mariadb-features is structured as a "What LLMs Often Miss" table whose rows reference subsequent sections ("use RETURNING", "use WITH SYSTEM VERSIONING", etc.). If the trap table is in docs/what-llms-miss.md and the corresponding explanations are in 10 separate files, the agent reading the trap table now has to follow links to understand what each recommendation means. The narrative breaks; the agent either fetches all of them (no win) or fetches none (worse output quality).
  3. Activation is decided by description, not by body size. The frontmatter description: is what the model reads to decide whether the skill is relevant. Splitting the body doesn't change that calculus at all — the skill either fires or it doesn't. So the "huge file in context" worry isn't even real: an unactivated skill of any size costs zero tokens.

The right granularity for an agent-consumed SKILL.md is ~8–12 KB — Robert's biggest is ~20 KB, well within sane limits. Anthropic's own published skills run in that band. The "many small files" pattern is a navigation aesthetic (good for humans browsing the repo) being misapplied as a runtime optimization (bad for agent ingestion).

What's worth salvaging

  • _shared/versioning.md has merit as an authoring convention — one source of truth for the "11.8 LTS baseline" sentence. But it should be inlined at build time (a small script that re-stamps the line into each SKILL.md) rather than loaded at runtime. Runtime sharing just adds another Read call.
  • A docs/ directory for genuinely rarely-referenced material (long worked examples, exhaustive sources lists) might make sense — but the headline trap tables, defaults-changed sections, and behavior-change warnings need to stay in SKILL.md.

Implementation defect, separate issue

.atl/skill-registry.md contains absolute file:///home/lorenzo-aps/Dev/skills/... paths — the PR author's laptop. That registry is useless to anyone else and needs to be fixed (or removed) before the PR could be merged on its own merits, independently of the modularization question.

How to communicate this back

Suggest the PR author run a quick A/B experiment: open a fresh Claude session with the unmodified mariadb-features skill, ask it three representative questions, count the turns and tokens. Then repeat with their modular version. The modular version will consistently take more turns (one Read per atomic file consulted) and more tokens once tool-use overhead is counted. That's the most convincing way to land the point without an architectural debate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants