Repository navigation
Add craft-cli: the craft developer CLI for plugins - #616
bigwolfeman wants to merge 105 commits into
Conversation
…raft apps Adds craft-command and its #[command] macro crate, craft-command-derive. craft-command holds a registry of commands that can be added and removed at run time, before/after hooks with per-owner time budgets, a change bus with a Tracked<T> wrapper, a bounded observer queue, JSON Schema from params docs, a JSON diff, and (feature testing) a conformance harness for apps. Part of storytold#414. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Adds craft-ipc, the transport for a Craft app that talks to other Craft apps on one machine. It is native only: interprocess has no wasm32 build. - server and client: one JSON object per line, push events, a bulk connection for frames, token auth on every connection, connection and time limits. - discovery: one record per running app in a per-user runtime directory, with a lock file that shows whether the owner is alive. - launch, trust, cache: find an app binary, start it only after an executable check, keep caches in a private per-user directory. - activate: focus hand-off between apps. - moniker, record, payload, asset, refresh, fetch, watch: asset names, stored link records, open payloads, linked-file polling and a frame cache. - PROTOCOL.md describes the wire protocol. The record schemas and fixtures are in the crate. - The crate compiles and passes clippy -D warnings on x86_64-pc-windows-gnu and aarch64-apple-darwin. This is a compile and lint fix only. No one ran the Windows or macOS code paths, so this does not claim they work there. Part of storytold#414. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
… gate Adds craft-plugin-abi. It describes a WebAssembly plugin package and decides whether the package may load, without running guest code. - manifest: the craft.manifest custom section, its parser and JSON Schema. - gate: check_package, the install-time policy check (size, memory, tables, functions, start function, imports, export signatures, feature allow list). - permissions, ops, descriptor, format, text: permission names, host call numbers and status codes, the craft_process descriptor, the format contribution shapes, and the rules for text a plugin supplies. - ABI.md lists the rules the crate encodes. Part of storytold#414. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
…browser) New crate craft-plugin-host with the engine layer. An Engine loads a package that craft-plugin-abi accepted and runs its exports under a budget (deadline, work, memory, stack, cancel). - wasmi engine on every target, fuel slices, deadline and cancel at slice edges - wasmtime engine behind feature `wasmtime` (native only) - browser engines behind feature `browser` (wasm32 only) - load_package: deadline rule, then the install gate, then compile - cross-engine conformance plugins and tests, README, workflow Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
ActivationBackend gains can_apply_token (default false). On Linux a token only counts when the backend can apply it to a live window. Stock winit cannot, so the answer is false plus a request for user attention. Wire values are unchanged. PROTOCOL.md says what each value means. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
… plugins Adds craft-plugin-sdk and craft-plugin-sdk-macros. They are the guest side of craft-plugin-abi. - host: call_json and typed helpers over craft_host.call (log, progress, cmd, state, doc, clock, random, link). - entry, ui: handler request and response types, and the panel tree builder. - rt, backend: the runtime behind the exports. The wasm32 modules are the only code that uses unsafe, each behind an explicit allow. Everything else is denied. - testing: a fake host, so handlers run in a plain cargo test. - #[craft_plugin(manifest = "craft.toml")] checks the manifest with the install gate's parser at compile time and emits the five ABI exports on wasm32. Part of storytold#414. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
- ChangeBus: each subscriber queue holds at most 256 changes. A full queue drops the oldest change and counts it. `drain_with_lost` returns the count; `drain` is unchanged. - ObserverQueue: the coalesce window is kept per id in `last_at`, so `drain` no longer resets it. The per-push scan of the call deque is gone. - json_diff: `Op.value` deserializes a present `"value": null` as `Some(Null)`, so a set to null stays a set after a serde round trip. An absent member stays `None`. - json_diff::patch: ops run on a copy, and the copy replaces the document only when every op succeeds. A failed patch leaves the document unchanged. - Interceptor::before takes `&Call`, not `&mut Call`. A gate can no longer edit params, id, origin or depth outside the verdict path, so the panic and budget rules cover every change. A compile_fail doctest pins this. Tests and docs follow the new signature. - Conformance T6: new `CommandHost::take_plugin_gate_ids`. T6 now checks that the plugin's own gate sees a probe command before unregister and sees nothing after it. It no longer relies on the host's veto-all gate. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
- manifest: a format contribution needs fs.picked in permissions.required or permissions.optional. Manifest::validate and the JSON Schema both enforce it. - gate: legacy v1 modules now run the feature allow list and module validation. - gate: the memory maximum uses checked_shl and saturating_mul. The reader already refuses an exponent of 64 or more, so this is hardening only. - format: SizeMismatch Display computes the byte count in u128. It overflowed u64 for u32::MAX by u32::MAX. - docs: ABI.md and README.md list what legacy modules skip and keep, and the format permission rule. - tests: overflow, page size, legacy feature, and fs.picked cases. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
…ssages A table with no declared maximum can grow past the 100,000 element cap through table.grow, so the gate now rejects it (TableMaxMissing), on both the native and the legacy path. Violation messages repeat import names, op names and serde text from the package, so Display now applies the same text rules as manifest text. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 # Conflicts: # Cargo.lock
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 # Conflicts: # Cargo.lock
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8 # Conflicts: # Cargo.lock
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
The gate now rejects a table without a declared maximum. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NFCDiYoKS325v2rtv2F7D8
# Conflicts: # Cargo.lock
# Conflicts: # Cargo.lock
# Conflicts: # Cargo.lock
# Conflicts: # Cargo.lock
check_package gathered facts before the size check, so a package over the cap with many craft.manifest sections still allocated an entry for each and could trigger the feature probes. An oversized package now returns PackageTooLarge alone, before any parsing.
A module that mixed ec_, pc_ and vc_ exports returned before the table and memory checks, so the author had to resubmit to learn about a table with no maximum. The caps now run first. The doc comment on check_package no longer claims an oversized package costs no allocation; it skips parsing and validation.
# Conflicts: # Cargo.lock # Cargo.toml # README.md
# Conflicts: # Cargo.lock # Cargo.toml # README.md
# Conflicts: # README.md
# Conflicts: # Cargo.lock # Cargo.toml
FredApps
left a comment
There was a problem hiding this comment.
Scoped review of CLI build/watch/dev-folder paths at b029d9ede01388901932f1df64ffe75167f20f8b; inherited SDK/host dependencies remain separately reviewed. No local builds or tests run; current hosted checks do not establish passing validation.
[P2] Reconcile ownership when a watched file changes plugin ID or is renamed. DevFolder::scan overwrites known[path].plugin_id after installing the new file without removing its previous_id. Saving plugin B over the path that previously held A leaves A loaded permanently, and deleting the path later removes only B. There is a related normal rename failure: rename a.wasm to b.wasm without changing its plugin ID; the scan installs b first, then sees a gone and removes that same ID, leaving b recorded as loaded while the plugin is actually absent (and its unchanged stamp prevents reload). Reconcile the full path-to-ID set before applying removals/replacements, or explicitly reject duplicate/changed ownership with safe handling. Add tests for both changing the manifest ID at one path and renaming a loaded file.
Remaining CLI subcommands and stack integration still need full review before approval.
Depends on #431, #432, #433, #454, #613 and #455. The branch sits on top of the host layer branch (#613), and one merge commit brings in the SDK branch (#455), so the Files tab also shows their commits until they merge. Review these two commits only:
craft-cli: the craft developer CLI for pluginsMerge pr/craft-plugin-sdk into pr/craft-cli, only to see how the conflicts were resolved.Cargo.tomlandREADME.mdkeep both sides.Cargo.lockkeeps the layer branch and gains the two SDK crates. The SDK code itself belongs to Add craft-plugin-sdk and craft-plugin-sdk-macros: the guest SDK for craft-abi 1 plugins #455.Part of #414.
What
A new crate,
craft-cli(crates/cli). It builds the programcraft. The library iscraft_cliand the binary is a thin wrapper, so every part has tests. The crate is native only: it runscargoand watches files withnotify. It has no wasm32 build and no wasm32 CI job.Size: 12 source files (1924 lines), 297 lines of plugin templates, 3 test files (815 lines) and one captured command list (226 lines).
The subcommands:
craft new <name> [--kind command,hook,effect,panel] [--dir DIR] [--id ID] [--sdk-path PATH]writes a plugin package:craft.toml,src/lib.rswith handlers and unit tests,Cargo.toml,.cargo/config.tomland aCargo.lock. The package depends oncraft-plugin-sdkby path. The default path is thecrates/plugin-sdkof the craft-libs checkout the binary was built from. Cargo fixes that path at build time, so a copied binary needs--sdk-path. The.cargo/config.tomlsets the linker flag for the wasm memory maximum, which the gate requires. The scaffold also has its own[workspace]table, so it builds the same inside or outside another repository.craft build [DIR] [--features F] [--out-name N]runscargo build --release --target wasm32-unknown-unknownin the package, runs the install gate on the module and prints the report. It writestarget/<name>.wasmonly when the gate accepts the module. A rejected build deletes an older file of that name. Exit code 1 means rejected.craft check <WASM>...runs the install gate on each file and prints one line each:ACCEPTED, orREJECTEDand one line per violation. Exit code 1 if any file is rejected or cannot be read. This is the subcommand for CI and scripts.craft inspect <WASM> [--json]prints the manifest, each op with the permission it needs, the permissions with their prompt level, the memory limits, imports, exports, custom sections and the gate result.craft run --headless <WASM> --invoke <ID> [JSON]installs the plugin into a realPluginManagerand calls one contribution. The host has no document and no app commands. A command goes through the command registry. A hook or panel gets the JSON as its request. An effect is refused with the reason, because headless has no image source.craft watch [DIR] [--install-into FOLDER]builds, then rebuilds after each burst of changes tosrc/,craft.tomlorCargo.toml. It never watchestarget/. It copies each accepted module into the folder under a hidden name, then renames it, so a reader never sees a half-written file.craft dev <FOLDER> [--invoke ID [JSON]]loads every.wasmin the folder into the headless host and replaces a plugin in place when its file changes. A file that fails the gate is reported and the old version keeps running. The watcher isdevfolder::DevFolder<T: DevTarget>, so an app can run it over its own plugin manager.craft bindgen --app NAME [--from FILE|-] [--out FILE]turns an app's command list into a Rust module of typed calls tocraft_plugin_sdk::cmd::execute. Without--fromit runs<NAME>-cli commands --json. The generator reads doc strings withcraft_command::schema::from_doc, the code the apps use.Changes from our staging copy
Interceptor::beforeorCommandHost::take_plugin_gate_ids, because it never runs an app.HeadlessServicesimplementsServicesas it is on the layer branch.check. Staging had no short, script-friendly gate command.../../crates/plugin-sdk) to../plugin-sdk, and its doc comment,--helptext and the README say that it is fixed at build time and meant for a binary built inside the checkout. The CLI has no other path that assumes the staging layout.Cargo.tomluses the workspace fields and[lints] workspace = true.craft-plugin-hostandcraft-cliare new entries in[workspace.dependencies], sincecraft-plugin-hostwas not listed yet.ABI.md. Aignoredoc fence becametext.craft inspectprintsmemory_mb 64, stack_kb unsetand notSome(64). The agent notes file was rewritten as a shortCLAUDE.md.wasm-ldwrites it with a maximum equal to the minimum (90 entries). I read the table section of the built module to confirm that. The e2e testbuild_makes_a_wasm_the_gate_acceptsfails if that ever changes. No fixture in this PR has a table, and the WAT plugins intests/devfolder.rshave none. The scaffold has noformatcontribution, so thefs.pickedrule does not apply to any template.Why
A plugin author needs a way to go from an empty folder to a module the install gate accepts, and to see a change running without an app. Before this PR the only path was to write the Cargo config, the manifest and the build command by hand, and to wait for an app to load the result. The CLI uses the same gate and the same plugin host as the apps (
craft_plugin_abi::check_package,PluginManager, the wasmi engine), so a pass incraft buildorcraft run --headlessis the same check an app makes.Dependencies
Workspace crates:
craft-command(#431),craft-plugin-abi(#432),craft-plugin-host(#613, with the engine PR #454). The scaffold needscraft-plugin-sdk(#455) at run time, and the e2e tests build it for wasm32.craft-ipc(#433) is a dependency of the host crate only.New third-party packages.
Cargo.lockgrows by 185 lines: the crate and 17 packages. Licenses read fromcargo metadata --format-version 1 --locked.clap4 (not in main's lock. Add craft-command: command registry, call hooks and change feed #431 adds it for a dev tool,MIT OR Apache-2.0),notify8.2.1 (CC0-1.0),getrandom0.2.17 (already locked,MIT OR Apache-2.0),wasmparser0.262.0 (not in main's lock. Add craft-plugin-abi: craft-abi 1 types, permission names and install gate #432 adds it,Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT). Dev:tempfile(in main's lock) andwat(not in main's lock. Add craft-plugin-abi: craft-abi 1 types, permission names and install gate #432 adds it).clap:clap_derive4.6.7,anstream1.0.0,anstyle-parse1.0.0,anstyle-query1.1.5,anstyle-wincon3.0.11,colorchoice1.0.5,is_terminal_polyfill1.70.2,once_cell_polyfill1.70.2, allMIT OR Apache-2.0.utf8parse0.2.2 isApache-2.0 OR MIT.strsim0.11.1 isMIT.notify:notify-types2.2.0 (MIT OR Apache-2.0),inotify0.11.5 andinotify-sys0.1.8 (ISC, Linux),kqueue1.2.1 andkqueue-sys1.1.2 (MIT, BSD and macOS),objc2-core-services0.3.2 (Zlib OR Apache-2.0 OR MIT, macOS).notifyis the one non-obvious license:CC0-1.0is a public domain dedication. There is nodeny.tomlin the repo, so I could not check it against a written policy. Say so in review if CC0 is not accepted. The alternative is a polling-only watcher, sincedevfolderalready rescans on a timer and the events only make it faster.Checks
Run on branch
pr/craft-cli, with-j4and one target directory. Test runs usedCARGO_PROFILE_DEV_DEBUG=0.cargo fmt --check -p craft-cli: exit 0.cargo clippy -p craft-cli --all-targets -j4 -- -D warnings: exit 0. The same with--target x86_64-pc-windows-gnuand--target aarch64-apple-darwin: exit 0.cargo clippy --workspace --all-targets -j4 --locked -- -D warnings: exit 0.cargo test -p craft-cli -j4 --locked: 42 passed, 0 failed, 0 ignored. By file: unit tests 5,tests/bindgen.rs8,tests/devfolder.rs6,tests/e2e.rs23, doc tests 0. The first e2e run took 21.7 s, which includes compiling the SDK for wasm32. The e2e tests run the realcraftbinary and real cargo./tmp), with the debug binary:craft new hello --kind command,hook,panel,effect:created ./hello (command, hook, panel, effect).craft build(withCARGO_BUILD_JOBS=4): cargoFinished release profile in 9.40s, thengate: ACCEPTED,plugin: dev.local.hello "hello" version 0.1.0,ops: 3 declared, 4 contributions,memory 0: initial 5 pages = 0.3 MiB, maximum 1024 pages = 64 MiB,wrote ./target/hello.wasm. The module is 152,187 bytes.craft check target/hello.wasm:target/hello.wasm: ACCEPTED, exit 0.craft run --headless target/hello.wasm --invoke say '{"name":"Ada"}':{"greeting": "hello, Ada"},[call end] completed.craft dev <folder> --invoke say '{"name":"Bo"}'loaded the plugin as generation 1 and printedhello, Bo.Not run:
.github/workflows/cli.ymlhas not run. It runs fmt, clippy and tests onubuntu-latestwith the wasm32 target installed, and it copies the action pins fromipc.ymlandplugin-sdk.yml.notifywith its inotify, kqueue and Windows backends, and only the inotify path has run here.craft watchby hand. The e2e testedit_rebuild_reload_loop_changes_what_the_plugin_answersdrives it: edit, rebuild, reload as generation 2, then a compile error that leaves generation 2 running. I did not run the subcommand in a terminal.craft bindgenagainst a live<app>-cli. The tests use a saved command list (tests/fixtures/filmcraft-commands.json, 45 commands captured from an app) and compile the generated module into a plugin for wasm32.cargo test --workspace, and--target wasm32-unknown-unknownfor the other crates. This PR changes no other crate. The rootCargo.tomlgets two lines and the lock file grows.craft-clifor wasm32. It cannot work, and no CI job asks for it. Existing CI jobs check one crate each (-p craft-<name>), so none builds this crate.craft newwith the default SDK path from an installed binary (cargo install). That case needs--sdk-path, and I did not run it.Not in this PR
craft bindgenfrom a running app over Craft Link, and running the data plane from the command line.devfolder::DevFolderis the piece they can reuse.craft-plugin-sdkto a registry. Until then the scaffold uses a path dependency.