Repository navigation
Add agent skills and AgentToolsObject bundles to DeployAgentTools - #248
Merged
Merged
Conversation
- AgentToolsObject: named bundles of MCP servers and agent skills (built-in
via $DefaultAgentTools, paclet-defined as "PacletName/Name", or ad hoc),
plus AgentToolsObjects for discovery
- InstallAgentSkills/UninstallAgentSkills: low-level, untracked skill
install/removal into a client's skills directory (OverwriteTarget support)
- $SupportedClients replaces $SupportedMCPClients (kept for back-compat) and
adds SkillsLocation/SkillsProjectPath for each client
- DeployAgentTools deploys bundles (MCP servers + skills) under a file lock
with rollback; deployment records move to schema v2 (a superset of v1)
- Skill registry reference-counts skill directories shared between
deployments/clients with baseline hashes; DeleteObject keeps skills that
are still referenced, pre-existing, or modified, with warnings
- Paclet extensions: "AgentSkills" spec (LLMSkill definitions or skill
directories) and multiple {"AgentTools", ...} entries per paclet
- Specs, docs, mock paclets, and tests for all of the above
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSX48r3Mie1gzVhZSjBDRz
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Symlink containment and Unix path matching can deploy skills from unintended locations.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Adds deployable agent-skill bundles alongside MCP servers, including client-specific installation, paclet discovery, reference-counted lifecycle management, and rollback.
Changes:
- Introduces
AgentToolsObject, skill installation, and tracked bundle deployment. - Extends paclet extensions and supported-client metadata for skills.
- Adds extensive validation, deployment, compatibility, and fixture coverage.
| File | Description |
|---|---|
Kernel/AgentSkills.wl |
Implements skill installation and registry management. |
Kernel/AgentToolsObject.wl |
Defines agent-tool bundles. |
Kernel/DeployAgentTools.wl |
Deploys bundles with locking and rollback. |
Kernel/SupportedClients.wl |
Adds client skill locations. |
Kernel/PacletExtension.wl |
Resolves paclet bundles and skills. |
Kernel/ValidateAgentToolsPacletExtension.wl |
Validates new extension forms. |
Kernel/InstallMCPServer.wl |
Adds shared preflight and removal helpers. |
Kernel/MCPServerObject.wl |
Supports multi-entry remote paclets. |
Kernel/CommonSymbols.wl |
Declares shared implementation symbols. |
Kernel/Files.wl |
Adds lock and registry paths. |
Kernel/Formatting.wl |
Formats bundles and deployed skills. |
Kernel/Main.wl |
Exports and loads new APIs. |
Kernel/Messages.wl |
Defines bundle and skill messages. |
PacletInfo.wl |
Registers new public symbols. |
Tests/AgentToolsObject.wlt |
Tests bundle APIs and properties. |
Tests/DeployAgentTools.wlt |
Updates all-client deployment tests. |
Tests/InstallMCPServer.wlt |
Tests config keys and preflight behavior. |
Tests/SupportedClients.wlt |
Tests skill locations and support flags. |
Tests/UninstallMCPServer.wlt |
Tests format-aware configuration removal. |
Tests/ValidateAgentToolsPacletExtension.wlt |
Tests bundle and skill validation. |
TestResources/MockMCPPacletSkills/PacletInfo.wl |
Declares valid fixture bundles. |
TestResources/MockMCPPacletSkills/NotARoot/foreign-skill/SKILL.md |
Provides an out-of-root fixture. |
TestResources/MockMCPPacletSkills/DevTools/Tools/DevTool.wl |
Defines a secondary-root tool. |
TestResources/MockMCPPacletSkills/DevTools/MCPServers/DevServer.wl |
Defines a secondary-root server. |
TestResources/MockMCPPacletSkills/DevTools/AgentSkills/dev-skill/SKILL.md |
Defines a secondary-root skill. |
TestResources/MockMCPPacletSkills/AgentTools/Tools/SkillsTool.wl |
Defines the primary fixture tool. |
TestResources/MockMCPPacletSkills/AgentTools/SkillSources/located-skill/SKILL.md |
Tests relative skill locations. |
TestResources/MockMCPPacletSkills/AgentTools/MCPServers/SkillsServer.wl |
Defines the primary fixture server. |
TestResources/MockMCPPacletSkills/AgentTools/MCPPrompts/SkillsPrompt.wl |
Defines a fixture prompt. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills/llmskill-skill.wl |
Tests LLMSkill definitions. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills/directory-skill/SKILL.md |
Tests directory skills. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills/directory-skill/scripts/run.wls |
Adds a bundled skill script. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills/directory-skill/references/guide.md |
Adds bundled reference material. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills/assoc-skill.wl |
Tests association-defined skills. |
TestResources/MockMCPPacletSkills/AgentTools/AgentSkills.wl |
Tests combined skill definitions. |
TestResources/MockMCPPacletConfigKey/PacletInfo.wl |
Declares the config-key fixture. |
TestResources/MockMCPPacletConfigKey/AgentTools/Tools/KeyTool.wl |
Defines its fixture tool. |
TestResources/MockMCPPacletConfigKey/AgentTools/MCPServers/KeyServer.wl |
Tests custom configuration keys. |
TestResources/MockMCPPacletBadSkills/PacletInfo.wl |
Declares invalid skill fixtures. |
TestResources/MockMCPPacletBadSkills/Second/AgentSkills/two-roots-skill.wl |
Tests duplicate roots. |
TestResources/MockMCPPacletBadSkills/AgentTools/Sources/wrong-name/SKILL.md |
Tests mismatched locations. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/two-roots-skill.wl |
Supplies the duplicate definition. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/not-a-skill.wl |
Tests invalid contents. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/no-description.wl |
Tests missing descriptions. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/mismatch-skill/SKILL.md |
Tests mismatched names. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/long-description.wl |
Tests description limits. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/dup-skill/SKILL.md |
Tests directory/file duplication. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/dup-skill.wl |
Supplies the duplicate file. |
TestResources/MockMCPPacletBadSkills/AgentTools/AgentSkills/Bad_Name.wl |
Tests invalid names. |
TestResources/MockMCPPacletBadBundles/PacletInfo.wl |
Declares invalid bundle fixtures. |
TestResources/MockMCPPacletBadBundles/AgentTools/Tools/GoodTool.wl |
Supplies a valid fixture tool. |
TestResources/MockMCPPacletBadBundles/AgentTools/MCPServers/ServerA.wl |
Defines a fixture server. |
TestResources/MockMCPPacletBadBundles/AgentTools/MCPServers/ServerB.wl |
Creates a config-key collision. |
Specs/PacletExtension.md |
Points to the superseding design. |
Specs/DeployAgentTools.md |
Marks the old skill design superseded. |
README.md |
Documents the new public APIs. |
docs/agent-tools-objects.md |
Adds the bundle and skill guide. |
docs/deploy-agent-tools.md |
Documents managed bundle deployment. |
docs/mcp-clients.md |
Documents client skill locations. |
docs/paclet-extensions.md |
Documents bundle and skill extensions. |
docs/preferences-content.md |
Clarifies the client registry source. |
docs/testing.md |
Documents the new fixtures. |
AGENTS.md |
Updates repository architecture guidance. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
CI runs the tests as root, where chmod doesn't prevent writes, so the rollback and remove-failed tests deployed successfully instead of failing. Use a regular file as the skills directory's parent for the rollback test and simulate the failed move for the remove-failed test; both now also run on Windows. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YSX48r3Mie1gzVhZSjBDRz
…rules - "SkillsDirectory" accepts only Automatic, None, or File[dir], as documented; a string (e.g. a mistyped "None") is no longer used as a directory - File[...] targets at a client's project path are compared case-folded only on Windows and macOS (foldPathCase, as for canonicalPathKey), so .VSCODE/mcp.json on Linux is not taken for VS Code's .vscode/mcp.json Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YSX48r3Mie1gzVhZSjBDRz
- Add descriptions to the built-in bundles - Show Name and Description in the visible summary rows; move MCP servers, agent skills, and location to the hidden rows - Truncate long descriptions to the summary panel width with a tooltip, falling back to the full description when no front end evaluates the truncation - List server and skill names in a Multicolumn and drop empty lists - Drop the MCP server icon from AgentToolsObject boxes for now Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0198aMp9hnyDsVchAs3ot6eM
Match MCPServerObject's "Tools"/"ToolNames": "MCPServers" now gives MCPServerObjects and "AgentSkills" gives LLMSkills, while "MCPServerNames"/"AgentSkillNames" give names. "AgentSkillNames" now keeps the qualified name of paclet skills. "Skills" is a new alias for "AgentSkills"; "MCPServerObjects" and "LLMSkills" remain as aliases. Internal callers that need the stored specifications (paclet installation before deploying, the deploy itself, and InstallAgentSkills/ UninstallAgentSkills on a bundle) now read them from "Data". Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3FPsx8u9QhpStd78vVsmD
- Build the complete skills into Assets/AgentSkills (shipped as the new "AgentSkills" asset); AgentSkills/ now holds only the sources. The build logic moved to the loadable Scripts/Resources/AgentSkillsBuilder.wl, and BuildAgentSkills.wls gained a --check mode. - Declare the built-in skills in $defaultAgentSkills (resolved from the loaded paclet at runtime) and add them to the $defaultAgentTools bundles. - For built-in bundles, DeployAgentTools no longer warns about targets without a skills directory, skips conflicting skill directories with AgentSkillNotInstalled instead of failing (unless nothing else could be deployed), and updates built-in skills installed by older versions. - Rewrite SetUpWolframMCPServer.md: prefer DeployAgentTools, document manual configuration for other clients, and describe the free Wolfram Cloud MCP service at https://agenttools.wolfram.com/mcp. - Add Tests/AgentSkillsBuild.wlt (staleness, packaging, builder) and tests for the built-in skills; isolate DeployAgentTools.wlt from the real AgentTools data directory; update the docs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DWiEUSezn4Zf23jwjzjXZc
- Without a front end (scripts, MCP servers), PacletCICD's CheckPaclet gives its hints as a UTF-8 encoded JSON string instead of a Dataset, which failed with an internal error. CheckPaclet and the pre-build check of BuildPaclet now decode it (checkHintsFromJSON). - CheckPaclet formats failures (e.g. a path without a definition notebook) like BuildPaclet instead of failing with an internal error. - extractFailureMessage fills in message parameters instead of returning the raw template, and explains a missing definition notebook instead of showing Missing["NotFound"]. The duplicate PacletCICD invfile messages are suppressed. - Convert private-use characters (e.g. \[Rule]) in check hint messages with convertPUACharacters, which is now declared in CommonSymbols.wl. - Add tests for JSON hints, private-use characters, failure messages, and tool calls on a paclet without a definition notebook. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LFCWoSrJE51KorM35LJYqo
- Disable progress reporting in the generated skill scripts, since progress output (e.g. "Connecting… | Elapsed time 1s" for slow requests) is written to stdout in scripts. - Convert private-use characters in the result written to stdout, like the MCP server does. The conversion is defined in the script, since the installed paclet version might not include it. - Rebuild the agent skills. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LFCWoSrJE51KorM35LJYqo
Add the wolfram-language, wolfram-notebooks, and wolfram-paclets skills to .claude/skills, deployed with DeployAgentTools from the built skills in Assets/AgentSkills. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LFCWoSrJE51KorM35LJYqo
ValidateAgentToolsPacletExtension accepted skill descriptions that are only whitespace, which deployment rejects in validateSkillDescription, so a paclet could validate successfully with a skill that cannot be deployed. Both checks now share agentSkillDescriptionQ. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgSwebQR2cWdUzwt3apHi2
ValidateAgentToolsPacletExtension accepted any value for the "SystemID" and "WolframVersion" qualifiers. "WolframVersion" must now be a string and "SystemID" must be All, a string, or a non-empty list of strings: the forms that PacletTools`PacletExtensions accepts (it rejects anything else, which sends getAgentToolsExtensions to the raw-extension fallback). The fallback and the duplicate-bundle-name check also treated the string "All" (and Automatic) as a wildcard, while PacletTools treats "All" as an ordinary system ID, so the same entry was filtered differently for installed and remote paclets. Both now use the PacletTools semantics. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SgSwebQR2cWdUzwt3apHi2
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The skills-only deployment accessor can return a malformed applied Missing expression, and the PR description also misstates generated-script compatibility.
Review effort: Balanced
Findings: 1
Open (2)
Resolved since last review (2)
Skills-only deployments have no primary MCP server, so the "LLMConfiguration" property applied the property to Missing["NotAvailable"] and returned an unevaluated expression. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DFLFbJkAP3iicbAF4QZBFW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds agent skills (
LLMSkill/ Agent Skills directories) as something AgentTools can discover and deploy alongside MCP servers, and ships the Wolfram agent skills fromAgentSkills/with the paclet as part of the built-in bundles. The design is in Specs/AgentToolsObject.md and the user guide is in docs/agent-tools-objects.md; the skill build is described in docs/agent-skills.md.New symbols (all in
Wolfram`AgentTools`)AgentToolsObject: a named bundle of MCP servers and agent skills. Bundles can be:$DefaultAgentTools, one bundle per default server, each with its built-in skills (see below);"PacletName/Name";Like
MCPServerObject's"Tools"/"ToolNames", the"MCPServers"and"AgentSkills"properties giveMCPServerObjects andLLMSkills, while"MCPServerNames"and"AgentSkillNames"give names (qualified names for paclet items)."Skills","LLMSkills"and"MCPServerObjects"are aliases. Summary boxes show the bundle's name and description.AgentToolsObjectslists bundles, with the same options asMCPServerObjects.InstallAgentSkills/UninstallAgentSkills: low-level and untracked.OverwriteTarget -> False, a different skill with the same name causes a failure; withTrue, it is overwritten.$SupportedClientsreplaces$SupportedMCPClients.$SupportedMCPClientsis kept for back-compat. It now lists only clients that have an MCPInstallLocation."SkillsLocation"and"SkillsProjectPath".~/.copilot/skillsand~/.agents/skillsare handled by the registry below.DeployAgentTools
MCPServerObject, or an association.*is never treated as a wildcard."SkillsDirectory"takesAutomatic,File[dir], orNone.DeleteObjectworks only from the recorded data and never resolves names again.OverwriteTarget:Truereplaces conflicting deployments and upgrades unmodified skills from the same source.Allalso overwrites modified or foreign skill directories.Skill registry (reference counting)
The registry is stored in
$deploymentsPath/.SkillRegistry. For each skill directory it records the referencing deployments and the baseline file hashes. Hashes are normalized for line endings.DeleteObjectremoves the deployment's reference. The directory is then deleted only when:AgentSkillInUse,AgentSkillNotRemovedwithUninstallAgentSkillsinstructions, orAgentSkillRemoveFailed).Paclet extensions
"AgentSkills"spec: LLMSkill definitions (.wl/.wxf/.mx) or skill directories.{"AgentTools", ...}entry implicitly defines a bundle named"PacletName/Name".Namedefaults to"AgentTools".Built-in agent skills
Build:
AgentSkills/now holds only the sources (Manifest.wl,References/, and one hand-authoredSKILL.mdper skill, without a version line).Scripts/BuildAgentSkills.wlswrites the complete skills (SKILL.mdwith the paclet version stamped in, generatedscripts/*.wls,references/) toAssets/AgentSkills/. That directory is committed and ships as the new"AgentSkills"paclet asset.Scripts/Resources/AgentSkillsBuilder.wl, so tests can use it.BuildAgentSkills.wls --checkreports a staleAssets/AgentSkills/without changing anything.Scripts/Resources/SkillScriptTemplate.wls:$ProgressReporting = False, so progress output no longer mixes into the tool result on stdout;\[Rule]) to ASCII, like the MCP server does.SetUpWolframMCPServer.mdchanged (rewritten, see below).GetWolframEngine.mdandScripts.mdare byte-identical..claude-plugin/marketplace.jsonplugins now use./Assets/AgentSkills, andwolfram-pacletsis added towolfram-language-development.Registry and bundles:
$defaultAgentSkillsdeclareswolfram-alpha,wolfram-language,wolfram-notebooksandwolfram-paclets. Each resolves at runtime to the loaded paclet's asset directory, so no build-machine path ends up in the MX.Wolframwolfram-language,wolfram-alphaWolframAlphawolfram-alphaWolframLanguagewolfram-language,wolfram-notebooks,wolfram-pacletsWolframPacletDevelopmentwolfram-language,wolfram-notebooks,wolfram-pacletsDeploying built-in bundles (other bundles are unchanged):
AgentSkillsNotDeployedwarning.DeployAgentTools[All, ...]omits them fromAgentSkillsNotDeployedWarningtoo.DeployAgentTools::AgentSkillNotInstallednames the skill, and the record lists it under the new"Skills"→"NotInstalled"key. If that would leave nothing to deploy, the conflict fails the deployment as before.DeployAgentTools[All, ...]summarizes withAgentSkillsNotInstalledWarning.OverwriteTarget, and the existingKeepNewerrule still prevents downgrading a shared directory.InstallAgentSkills, ad hoc bundles, and the"AgentSkills"property.SetUpWolframMCPServer.md(copied into every skill) is rewritten:DeployAgentTools, usingInstallMCPServeronly when just the MCP server is wanted (e.g. when the skills are already installed as a Claude Code plugin).https://agenttools.wolfram.com/mcp(no subscription or API key), with links to the docs and the support article.SKILL.mdexamples now use the actual MCP config key (mcp__Wolfram__...).Notes / known limitations
Preferences UI: it deploys the built-in bundles, so its Configure button now also installs the built-in skills for clients that support them.
Manual skills build: after changing a skill source, a reference, or a tool used by a skill, run
wolframscript -f Scripts/BuildAgentSkills.wlsand commitAssets/AgentSkills/. A test fails when the committed skills are stale.Version lag: the
metadata.versionin the builtSKILL.mdfiles lags the paclet version after CI's automatic version bumps. Upgrade decisions use the paclet version, not this field.User docs: the quick-start pages and tutorial notebooks still show
InstallMCPServer. They are accurate but don't mentionDeployAgentTools.No documentation notebooks yet for the new symbols. Markdown docs and specs are included.
Tested on Linux only. The Windows and macOS skill paths come from client documentation and were not exercised.
Bundles don't filter on
WolframVersionqualifiers.An unreadable subdirectory inside a skill source is skipped silently.
Upstream LLMFunctions issues found along the way:
"allowed-Tools"key typo;LLMSkill[File[dir]]fails on Latin-1 accented frontmatter.AgentTools works around the second one with its own
SKILL.mdparser fallback.Test plan
Tests/AgentSkills.wltTests/AgentToolsObject.wltTests/DeployAgentToolsSkills.wlt, which covers paclet bundles, remote paclet stubs, shared and symlinked directories, rollback, v1 records,OverwriteTarget -> All, accented frontmatter, and the built-in bundles (no-skills targets,Allmode, conflicting and modified skills, the nothing-left-to-deploy case, and updates on a shared root).Tests/AgentSkillsBuild.wlt: fails if the committed skills are stale, plus checks of the paclet asset, source hygiene,marketplace.json, the setup reference, and the builder functions.DeployAgentTools(now isolated from the real AgentTools data directory),InstallMCPServer,UninstallMCPServer,SupportedClients,PacletExtension,ValidateAgentToolsPacletExtension.SubmitUsagePayload-Endpoint, which POSTs to the usage endpoint. It fails identically on the previous commit because the endpoint now redirects to a login page, and CI skips it.AgentSkills,AgentSkillsBuild,AgentToolsObject,DeployAgentTools,DeployAgentToolsSkills) pass 617/617..pacletcontains all 28 skill files byte-identical to the checkout, and the affected suites pass against the built MX. At runtime, the built paclet's skills resolve to its own asset directory.wolframscript -f Scripts/BuildAgentSkills.wls --checkexits 0..wl/.wltfiles.~/.copilot/skillsare kept after deleting the first deployment and removed with the last.InstallAgentSkills/UninstallAgentSkillsround-trip.DeployAgentTools["ClaudeCode", "Wolfram"]installs the MCP server pluswolfram-languageandwolfram-alphawith the paclet version stamped in. The previous commit installed no skills.AgentSkillNotInstalled.🤖 Generated with Claude Code
https://claude.ai/code/session_01DWiEUSezn4Zf23jwjzjXZc