feat: add durable identity-validated microVM cleanup - #7896
Conversation
Persist identity-validated Cloud Hypervisor cleanup records. Safely reap stale resources after abrupt owner death. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
🚀 Security Guard has started processing this pull request |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Process identity can misclassify active runs after binary replacement, and iptables deletion failures can discard recovery records.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review tier: Balanced
Findings: 1
New issues introduced by this change (2)
| Severity | Finding |
|---|---|
src/cloud-hypervisor/cleanup-registry.ts — The executable identity is taken from the resolved pathname rather than from the running executable… |
|
src/microvm/network-commands.ts — This treats every iptables deletion failure as successful. MicrovmNetworkManager.cleanup() then… |
What changed in this PR
Adds durable, identity-validated recovery for Cloud Hypervisor resources after abrupt termination.
Changes:
- Persists and reaps per-run cleanup records.
- Tracks network, process, cgroup, mount, and directory identities.
- Adds unit and live-KVM recovery coverage.
| File | Description |
|---|---|
src/microvm/network.ts |
Exports the resource observer type. |
src/microvm/network.test.ts |
Tests resource observation and tagged rules. |
src/microvm/network-types.ts |
Defines rule comments and resource observers. |
src/microvm/network-plan.ts |
Generates per-run rule comments. |
src/microvm/network-manager.ts |
Reports created resources and uses tagged rules. |
src/microvm/network-commands.ts |
Adds comments to iptables rules. |
src/cloud-hypervisor/vm-config-builder.test.ts |
Updates network-plan fixture. |
src/cloud-hypervisor/virtiofsd.ts |
Records virtiofsd process identities. |
src/cloud-hypervisor/manager.ts |
Integrates the cleanup registry. |
src/cloud-hypervisor/manager.test.ts |
Tests cleanup-record lifecycle. |
src/cloud-hypervisor/manager-types.ts |
Extends manager dependency contracts. |
src/cloud-hypervisor/manager-stop.ts |
Completes records after teardown. |
src/cloud-hypervisor/manager-start.ts |
Reaps stale records and captures resources. |
src/cloud-hypervisor/cleanup-registry.ts |
Implements durable validated cleanup. |
src/cloud-hypervisor/cleanup-registry.test.ts |
Tests registry recovery behavior. |
scripts/ci/cloud-hypervisor-live-smoke.sh |
Adds abrupt-death recovery coverage. |
docs/cloud-hypervisor-foundation.md |
Documents durable cleanup semantics. |
.github/workflows/test-cloud-hypervisor.yml |
Detects namespace and rule residue. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 93.86% | 92.98% | 📉 -0.88% |
| Statements | 92.67% | 91.62% | 📉 -1.05% |
| Functions | 93.24% | 92.15% | 📉 -1.09% |
| Branches | 86.14% | 84.87% | 📉 -1.27% |
📁 Per-file Coverage Changes (7 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/microvm/network-commands.ts |
97.8% → 96.0% (-1.77%) | 97.9% → 96.2% (-1.69%) |
src/cloud-hypervisor/manager-start.ts |
98.2% → 97.0% (-1.25%) | 98.3% → 95.7% (-2.65%) |
src/microvm/network-manager.ts |
98.4% → 98.5% (+0.11%) | 96.9% → 97.1% (+0.22%) |
src/cloud-hypervisor/manager-stop.ts |
94.6% → 94.7% (+0.14%) | 89.1% → 89.5% (+0.42%) |
src/cloud-hypervisor/manager.ts |
88.9% → 89.1% (+0.21%) | 87.5% → 87.7% (+0.21%) |
src/cloud-hypervisor/virtiofsd.ts |
77.7% → 78.3% (+0.58%) | 74.8% → 75.5% (+0.62%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
✨ New Files (1 files)
src/cloud-hypervisor/cleanup-registry.ts: 65.4% lines
Coverage comparison generated by scripts/ci/compare-coverage.ts
Capture executable inode identity from the live procfs object. Verify firewall rule absence after deletion. Cover cleanup races at base coverage levels. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92d2c346-d396-40d3-8b74-1f1447a04871
|
✅ Copilot review passed with no inline comments. @lpcox Add the |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (9 files)
✨ New Files (1 files)
Coverage comparison generated by |
Resolve Cloud Hypervisor lifecycle overlaps. Preserve durable cleanup and current runtime confinement. Assign the VMM cgroup before durable identity capture. This avoids a PID-reuse window. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92d2c346-d396-40d3-8b74-1f1447a04871
Ensure VMM cgroup assignment precedes durable process identity capture. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 92d2c346-d396-40d3-8b74-1f1447a04871
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.
|
|
📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤
|
|
✅ Contribution Check completed successfully! PR #7896 follows CONTRIBUTING.md sufficiently for this review. The description is clear, references the change and validation, the new functionality includes tests, documentation is updated, and the new files are placed in the expected
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅
|
|
✅ Smoke Claude passed Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "msfeed25.pkgs.visualstudio.com"
- "registry.npmjs.org"See Network Configuration for more information.
|
|
✅ Build Test Suite completed successfully!
|
|
🚀 Security Guard has started processing this pull request |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "msfeed25.pkgs.visualstudio.com"
- "registry.npmjs.org"See Network Configuration for more information.
|
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
|
🚀 Security Guard has started processing this pull request |
|
❌ Smoke Gemini reports failed. Facets need polishing...
|
|
✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓
|
|
✅ Build Test Suite completed successfully!
|
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓
|
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅
|
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓
|
|
🔌 Smoke Services — All services reachable! ✅
|
|
❌ Contribution Check failed. Please review the logs for details.
|
|
EGRESS_RESULT allow=pass deny=pass ✅ Allowed domain (github.com) reachable: Overall: PASS cc @lpcox Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
Smoke Test: Claude Engine Validation
Overall result: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "api.anthropic.com"See Network Configuration for more information.
|
|
Smoke Test: Copilot Engine — @lpcox
Overall: PASS
|
Chroot Version Comparison Results
Result: Not all tests passed (Node.js version mismatch between host and chroot). The
|
Smoke Test
Warning Firewall blocked 2 domainsThe following domains were blocked by the firewall during workflow execution:
To allow these domains, add them to the network:
allowed:
- defaults
- "msfeed25.pkgs.visualstudio.com"
- "registry.npmjs.org"See Network Configuration for more information.
|
|
Docker Sbx Smoke Test — @lpcox
Overall: PASS
|
|
Smoke Test: GitHub Actions Services Connectivity
Overall: FAIL —
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — PASS Notes:
|
|
Smoke Test: Copilot BYOK (Direct Mode) ✅ PASS
Running in direct BYOK mode (
|
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra Overall: FAIL
|
|
feat: isolate Cloud Hypervisor with per-run VMM identities: ❌
|
|
OTel Tracing Smoke Test Results
All scenarios passed. No regressions detected in OTEL tracing integration.
|


Summary
Validation
npm test -- --runInBand(330 suites, 5284 tests)npm run buildnpx markdownlint-cli2 docs/cloud-hypervisor-foundation.mdbash -n scripts/ci/cloud-hypervisor-live-smoke.sh