feat(sync): add plugin system for extensible post-deployment sync steps#513
Draft
feat(sync): add plugin system for extensible post-deployment sync steps#513
Conversation
…erface - Add `type: plugin` as a new sync step type in canister.yaml (path/url/sha256/dirs fields) - New `crates/icp-sync-plugin` crate: sandbox path enforcement implemented and tested; runtime stub pending wasmtime Component Model implementation - Wire SyncStep::Plugin through manifest adapter, syncer, deploy and sync commands - Define plugin interface in sync-plugin/sync-plugin.wit (WIT / Component Model) - Add design.md and plan.md in sync-plugin/ - Add POC plugin skeleton in sync-plugin/poc/ - Update JSON schemas Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… plugin Replace the stub in `crates/icp-sync-plugin/src/runtime.rs` with a full wasmtime component model host implementation. The host provides four import functions to the plugin (canister-call, read-file, list-dir, log) and calls the plugin's exported exec() function. Also flesh out the proof-of-concept guest plugin in sync-plugin/poc/ with wit-bindgen bindings and a seed-data uploader that demonstrates the full host↔guest contract end-to-end. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add examples/icp-sync-plugin with a Rust CDK canister that stores (name, content) pairs seeded by the sync plugin from seed-data/ files - Add Candid interface (demo.did) and ic-wasm step to embed it - Link WASI P2 in the wasmtime host so wasm32-wasip2 plugins work - Walk the full error cause chain in sync failure output for better error messages; include wasm path in ReadWasm error - Update POC plugin to pass (filename, content) to the canister's seed() Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the custom `read-file` / `list-dir` / `stat` WIT imports with WASI preopens of the manifest's `dirs` entries, so plugins traverse them with standard `std::fs`. Add a new `files` manifest field whose contents the host reads and passes inline via `sync-exec-input`. Update the example canister (`set_config`, `register`, `show`) and POC plugin to exercise both paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
canister-call now exchanges raw Candid-encoded bytes in both directions. The host forwards arg bytes to ic-agent and returns the response bytes unchanged; plugins are responsible for encoding arguments and decoding responses. The POC plugin is updated accordingly and trimmed to just set_config + register. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Highest wasmtime version compatible with Rust 1.90.0 (42+ requires 1.91.0). Adapts to API breakage: WasiView::ctx now returns WasiCtxView, add_to_linker_sync moved to the p2 module, and component bindgen requires a HasData marker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidate target/ ignore rule into root .gitignore so nested Rust workspaces don't each need their own gitignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…Pipe Drops LineBuf/PluginStdio/PluginOutputStream and the host-side log() import in favour of MemoryOutputPipe: plugin stdout/stderr are captured after exec() returns and forwarded to the progress channel. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keeps the WIT file alongside its host-side implementation rather than in a separate top-level directory. Update all path references in build.rs and bindgen! / wit_bindgen::generate! invocations accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix the sync-plugin.wit link to its new location in the crate. Update the stdio section: output is now buffered until exec() returns (stdout then stderr), removing stale line-buffering / 64 KiB details. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…workspace Combined the sync plugin PoC (sync-plugin/poc/) and the example canister into a single Cargo workspace under examples/icp-sync-plugin/. The canister lives in canister/ and the plugin in plugin/. Updated icp.yaml and WIT paths accordingly; removed sync-plugin/poc from the root workspace excludes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes the stale top-level sync-plugin/ directory (design.md, plan.md) and SANDBOX.md, replacing them with a DESIGN.md that reflects the current wasmtime/WASI-based implementation and a TODO.md with remaining work items. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…canister-call Threads args.proxy from icp deploy through sync_many → Params so plugin syncers can route update calls via the proxy canister. Extends the WIT interface with a direct: bool field on canister-call-request; when true the host bypasses the proxy and calls the target canister directly even if --proxy was set. The assets and script syncers are unaffected. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y principal - Add identity-principal and proxy-canister-id to sync-exec-input in the WIT interface so plugins can act on the caller's identity and proxy configuration. - Replace set_config with set_uploader(Principal): controller-gated update that stores the uploader; register is now restricted to that principal. - Plugin calls set_uploader via proxy (direct: false) and register directly (direct: true), demonstrating both routing modes in a single sync run. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers project structure (canister, plugin, seed-data), the role of each component, and a walkthrough of how the direct flag is exercised across the two canister calls made during a sync run. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wasmtime 43 requires Rust 1.91.0 (MSRV bump) and changed its error type from anyhow::Error to wasmtime::Error. Update snafu source fields in icp-sync-plugin accordingly and drop the now-unused anyhow dependency. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a sync plugin system that lets users extend
icp syncwith arbitrary post-deployment logic via sandboxed WebAssembly components, without shelling out or modifying the CLI.For full design rationale, architecture decisions, and interface specification, see crates/icp-sync-plugin/DESIGN.md.
New crate:
icp-sync-pluginA new
crates/icp-sync-plugincrate provides the Wasmtime-based Component Model runtime that loads and executes plugin.wasmfiles. Key pieces:sync-plugin.wit— the WIT interface defining what a plugin can call on the host (canister update/query calls, file reads via WASI preopens) and what the host invokes on the plugin (exec())src/runtime.rs— the wasmtime host implementation: component instantiation, WASI sandbox setup, host function bindings, and executionbuild.rs— WIT bindings generation at build timePlugins run in a WASI sandbox with capability-based access: they can only reach the single canister being synced and only read from explicitly declared directories.
New example:
examples/icp-sync-pluginA self-contained workspace demonstrating the full flow end-to-end:
canister/— a Rust CDK canister that accepts uploaded data via update callsplugin/— awasm32-wasip2plugin that reads seed files from a preopened directory and calls the canister to upload themicp.yaml— project manifest wiring the plugin step to the canister withtype: plugin, declareddirs, and an optionalsha256Adjustments to the existing codebase
crates/icp/src/manifest/): newpluginadapter variant inadapter/plugin.rs;canister.rsextended withPluginStepfields (path,url,sha256,dirs,args,env)crates/icp-cli/src/operations/sync.rs): plugin step dispatch added alongside existingscriptandassetssteps;--proxyflag forwarded through to the runtime so plugins can reach replicas behind a proxyicp synccommand (crates/icp-cli/src/commands/sync.rs):--proxyflag exposed on the CLIdocs/schemas/): regenerated to include the newpluginstep fieldsTest plan
cargo build --bin icpexamples/icp-sync-plugin/README.mdcargo testpassescargo fmt && cargo clippyare clean🤖 Generated with Claude Code