v0.11.2: tokio runtime drop panic on every local-path worker VM boot after #1514

[rohitg00]

Repro

iii update                # iii-worker 0.10.0 -> 0.11.2 (post #1514)
cd <any project with local-path workers in config.yaml>
iii                       # engine boots, local workers all 'stopped'
iii worker logs <name>

Symptom

Every local-path worker panics at VM-boot time:

  Booting VM (2 vCPUs, 2048 MiB RAM)...

thread 'main' panicked at /Users/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.0/src/runtime/blocking/shutdown.rs:51:21:
Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.

iii-observability, iii-state, iii-queue (config workers) still come up clean. Only local-path workers that boot through iii-worker __vm-boot are affected.

Environment

• Host: macOS 25.2.0 darwin-arm64
• iii: 0.11.2 (commit f38f0ea / PR feat(iii-worker): local-path worker image override + shell-exec + OCI fixes #1514)
• libkrun firmware: libkrunfw-darwin-aarch64.dylib bundled with iii-worker

Suspected cause

Something in crates/iii-worker/src/cli/vm_boot.rs or one of the new shell-relay / supervisor modules introduced by #1514 is dropping a multi-threaded tokio runtime while still inside an async task, which tokio 1.52 explicitly forbids.

Impact

Blocks every iii project that boots local-path workers on v0.11.2. Reverting to 0.11.0 would restore function but doesn't include the fixes from #1514.

[rohitg00]

Still reproduces on v0.11.2 — 2026-04-22

Fresh repro from a live iii-experimental/roster session. Bug persists two months after filing.

Environment delta

• Host: macOS 26.2 (build 25C56), darwin-arm64 — M-series (T6000)
• iii: 0.11.2 (same)
• libkrunfw: ~/.iii/lib/libkrunfw.5.dylib (22MB)
• Config: 20 local-path workers (roster + phase-2 narrow workers)

New symptom details

The original report captured the tokio panic via iii worker logs <name>. On this box, the panic trace is no longer surfacing at all:

• iii-worker child processes exit immediately on spawn → become zombies
• Engine shows 21 defunct children of the engine pid
• /tmp/iii-worker-logs/<worker>/ directory is never created (so iii worker logs <name> returns empty)
• ~/.iii/managed// is prepared but vm.pid / watch.pid point at dead pids from a prior boot; clearing them does not help

$ ps ax -o pid,ppid,state,command | awk -v p=$ENGINE_PID '$2==p'
34195 34193 Z    <defunct>
34196 34193 Z    <defunct>
... (21 total)

Standalone iii-worker invocation produces zero output

Trying to run iii-worker directly with full debug env produces no stderr at all:

III_URL=ws://localhost:49134 \
III_ENGINE_URL=ws://localhost:49134 \
WORKER_NAME=issues \
RUST_LOG=trace \
RUST_BACKTRACE=full \
iii-worker > /tmp/wk.out 2> /tmp/wk.err < /dev/null

$ wc -l /tmp/wk.out /tmp/wk.err
0 /tmp/wk.out
0 /tmp/wk.err

No panic trace, no log line. Process either exits 0 silently or is killed before the tokio panic handler flushes to stderr. Smells like the panic is happening in a detached / early-drop path where the runtime itself never gets to print.

Impact update

Every iii-experimental project with worker_path: entries in config.yaml is blocked on v0.11.2 at libkrun cold-boot. roster, workers-registry testing, anything with Node/Rust local workers.

Any ETA on a fix or a supported isolation: none switch in iii.worker.yaml? Strings in the binary already reference III_ISOLATION with value none — setting it doesn't change behavior though, still zombies.

[rohitg00]

I guess this is fixed, correct? @ytallo and @andersonleal