Skip to content

feat: bind Cloud Hypervisor artifacts to attested release manifests - #7891

Merged
lpcox merged 3 commits into
mainfrom
lpcox-signed-artifact-manifest
Aug 31, 2026
Merged

feat: bind Cloud Hypervisor artifacts to attested release manifests#7891
lpcox merged 3 commits into
mainfrom
lpcox-signed-artifact-manifest

Conversation

@lpcox

@lpcox lpcox commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • publish one release-pinned manifest covering Cloud Hypervisor, virtiofsd, kernel, rootfs, and supervisor versions, canonical filenames, and SHA-256 digests
  • attest that manifest with GitHub artifact attestations and ship its Sigstore bundle as a release asset
  • verify the bundle against github/gh-aw-firewall and the exact release workflow before parsing the manifest or trusting any contained hash
  • require the manifest release to match the running AWF version, then verify all five local artifacts and propagate the verified supervisor digest into rootfs preparation
  • retain legacy caller-provided hashes only behind a preview-only dual development opt-in

Threat model

This closes the substitution gap where a caller could provide a malicious artifact together with its matching SHA-256 value. Trust now starts at GitHub OIDC/Sigstore provenance for the protected release workflow. Preflight fails closed on an invalid signer, self-hosted signer, malformed or stale manifest, renamed artifact, unsafe ownership/permissions, or digest mismatch.

The boundary still trusts host root, the locally resolved root-owned gh executable, GitHub's attestation/Sigstore trust chain, and the protected release workflow itself. The local bundle avoids GitHub API lookup; gh may need network access to initialize or refresh Sigstore trust-root material unless it is already cached or provisioned.

Migration

Release users replace the five --cloud-hypervisor-*-sha256 trust arguments with:

--cloud-hypervisor-artifact-manifest /trusted/manifest.json
--cloud-hypervisor-artifact-manifest-bundle /trusted/manifest.sigstore.jsonl
--cloud-hypervisor-artifact-release-tag vX.Y.Z

The tag must match the AWF binary version. Same-run development CI can use --cloud-hypervisor-development-allow-unattested-artifacts only together with AWF_CLOUD_HYPERVISOR_DEVELOPMENT_ALLOW_UNATTESTED_ARTIFACTS=1 and all five legacy digests.

Validation

  • npm test -- --runInBand (330 suites, 5,283 tests)
  • npm run type-check -- --pretty false
  • npm run build
  • release workflow YAML parse
  • Bash syntax checks for artifact build/verification and live smoke scripts
  • git diff --check

Verify release-pinned GitHub provenance before trusting microVM artifact hashes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 31, 2026 14:35
@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation build failed for this PR. View logs.

Built from commit c27ff2d

@lpcox lpcox changed the title Bind Cloud Hypervisor artifacts to attested release manifests feat: bind Cloud Hypervisor artifacts to attested release manifests Aug 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.86% 93.77% 📉 -0.09%
Statements 92.67% 92.58% 📉 -0.09%
Functions 93.24% 93.21% 📉 -0.03%
Branches 86.14% 86.02% 📉 -0.12%
📁 Per-file Coverage Changes (4 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/runtime-validation.ts 100.0% → 95.8% (-4.17%) 100.0% → 95.8% (-4.17%)
src/cloud-hypervisor/preflight.ts 96.2% → 94.4% (-1.76%) 95.7% → 93.5% (-2.20%)
src/cloud-hypervisor-runtime-backend.ts 96.3% → 96.3% (+0.01%) 93.5% → 93.5% (+0.02%)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)
✨ New Files (1 files)
  • src/cloud-hypervisor/artifact-manifest.ts: 79.4% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

User-owned manifests and artifacts can be replaced between verification and consumption, enabling provenance-bypassing TOCTOU races.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 2 High severity

New issues introduced by this change (2)
Severity Finding
High severity src/​cloud-hypervisor/​preflight.ts — Reopening the manifest by path after gh exits creates a TOCTOU bypass. assertTrustedRegularFile
High severity src/​cloud-hypervisor-runtime-backend.ts — This propagates verified digests but keeps all original caller-controlled paths. Preflight permits…
What changed in this PR

Adds Sigstore-attested release manifests as the trust root for Cloud Hypervisor preview artifacts.

Changes:

  • Adds manifest configuration, validation, parsing, and digest propagation.
  • Retains legacy hashes behind a dual development opt-in.
  • Builds, attests, publishes, tests, and documents release manifests.
File Description
src/​types/​runtime-options.ts Extends runtime artifact options.
src/​config-mapper.ts Maps manifest configuration to CLI options.
src/​config-file.ts Adds manifest configuration fields.
src/​commands/​build-config.ts Builds manifest-aware runtime configuration.
src/​cloud-hypervisor/​runtime-validation.ts Enforces release or development trust modes.
src/​cloud-hypervisor/​runtime-validation.test.ts Tests trust-mode validation.
src/​cloud-hypervisor/​preflight.ts Verifies attestations and artifact digests.
src/​cloud-hypervisor/​preflight.test.ts Tests attestation preflight behavior.
src/​cloud-hypervisor/​config.test.ts Tests configuration mapping.
src/​cloud-hypervisor/​artifact-manifest.ts Defines and validates manifests.
src/​cloud-hypervisor/​artifact-manifest.test.ts Tests manifest validation.
src/​cloud-hypervisor-runtime-backend.ts Propagates verified digests to the manager.
src/​cloud-hypervisor-runtime-backend.test.ts Tests digest propagation.
src/​cli-options.ts Adds manifest and bypass flags.
src/​awf-config-schema.json Extends the generated schema.
scripts/​ci/​cloud-hypervisor-live-smoke.sh Enables the CI development bypass.
scripts/​ci/​cloud-hypervisor-ci-scripts.test.ts Tests manifest CI integration.
README.md Updates the preview feature summary.
guest/​cloud-hypervisor/​verify-test-artifacts.sh Verifies generated manifest entries.
guest/​cloud-hypervisor/​build-test-artifacts.sh Generates the release manifest.
docs/​cloud-hypervisor-foundation.md Documents trust and migration behavior.
docs/​awf-config.schema.json Updates the public configuration schema.
docs/​awf-config-spec.md Documents manifest configuration.
.github/​workflows/​release.yml Attests and publishes the manifest bundle.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/cloud-hypervisor/preflight.ts Outdated
Comment thread src/cloud-hypervisor-runtime-backend.ts
…-manifest

# Conflicts:
#	src/cloud-hypervisor/runtime-validation.test.ts
@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@lpcox Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

Prevent manifest and runtime artifact replacement between provenance verification and consumption.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 38ffbe47-fa3c-4a37-b677-dec7d83ecc50
@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Coverage Regression Detected

This PR decreases test coverage. Please add tests to maintain coverage levels.

Overall Coverage

Metric Base PR Delta
Lines 93.82% 93.72% 📉 -0.10%
Statements 92.60% 92.51% 📉 -0.09%
Functions 93.23% 93.20% 📉 -0.03%
Branches 85.97% 85.86% 📉 -0.11%
📁 Per-file Coverage Changes (4 files)
File Lines (Before → After) Statements (Before → After)
src/cloud-hypervisor/runtime-validation.ts 100.0% → 96.1% (-3.93%) 100.0% → 96.1% (-3.93%)
src/cloud-hypervisor/preflight.ts 96.2% → 94.4% (-1.76%) 95.7% → 93.5% (-2.20%)
src/cloud-hypervisor-runtime-backend.ts 96.3% → 96.3% (+0.01%) 93.5% → 93.5% (+0.02%)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)
✨ New Files (1 files)
  • src/cloud-hypervisor/artifact-manifest.ts: 79.4% lines

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK reports failed. BYOK mode investigation needed...

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

Generated by Build Test Suite for #7891

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude failed

Generated by Smoke Claude for #7891

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Contribution Check completed successfully!

Contribution review complete: PR #7891 appears to follow CONTRIBUTING.md. The change includes tests for the new manifest/attestation flow, updates relevant documentation and schema/help text, and places new code/configuration in the expected directories.

Generated by Contribution Check for #7891

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • msfeed25.pkgs.visualstudio.com
  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "msfeed25.pkgs.visualstudio.com"
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

📰 DEVELOPING STORY: Smoke Docker Sbx reports failed. Our correspondents are investigating the incident...

📰 BREAKING: Report filed by Smoke Docker Sbx

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Aug 31, 2026
@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (github.com) reachable: allowed=200
✅ Blocked domain (example.com) denied: CONNECT tunnel failed, 403

Overall status: PASS

cc @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

fix: filter workflow commands from microVM output
feat: verify Cloud Hypervisor runtime confinement
GitHub MCP connectivity: ✅
GitHub.com HTTP: ✅
File I/O: ✅
BYOK inference: ✅
Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra
Overall: PASS
@lpcox

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results:

  • Redis PING: ❌ (Temporary failure in name resolution)
  • PostgreSQL pg_isready: ❌ (no response)
  • PostgreSQL SELECT 1: ❌ (could not translate host name)

Overall: FAILhost.docker.internal does not resolve in this sandbox, so no service connectivity could be verified.

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

@lpcox

  • fix: filter workflow commands from microVM output: ❌
  • feat: verify Cloud Hypervisor runtime confinement: ✅
  • GitHub.com connectivity: ✅
  • File I/O test: ✅
  • BYOK inference: ✅
    Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)
    Overall: FAIL

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine

Overall: PASS

cc @lpcox

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color ok ✅ PASS
Go env ok ✅ PASS
Go uuid ok ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

All 18 test projects across Bun, C++, Deno, .NET, Go, Java, Node.js, and Rust built/installed and passed their tests successfully. All repositories cloned without issue.

Notes:

  • Java required -Dmaven.repo.local=<writable path> since ~/.m2/repository was not writable in this environment (pre-existing directory owned by a different user); this is an environment quirk, not a firewall issue.

Generated by Build Test Suite for #7891 · copilot · auto · 34.6 AIC · ⊞ 12K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: API Proxy OTEL Tracing — all scenarios pass

  • ✅ Scenario 1 (Module Loading): otel.js loads cleanly, exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, plus internal helpers.
  • ✅ Scenario 2 (Test Suite): otel.test.js, otel-fanout.test.js, otel-workload-identity.test.js, token-tracker-http.unit.test.js → 4 suites passed, 99/99 tests passed.
  • ✅ Scenario 3 (Env Var Forwarding): env-passthrough.ts forwards GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID (+ COPILOT_OTEL_FILE_EXPORTER_PATH) into the agent; api-proxy-env-config.ts forwards OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, and the same trace/span env vars into the api-proxy.
  • ✅ Scenario 4 (Token Tracker Integration): token-tracker-http.js calls the onUsage callback after normalized usage extraction — confirmed hook point for OTEL span attribute updates (gen_ai.usage.input_tokens/output_tokens set in otel.js).
  • ⚠️ Scenario 5 (OTEL Diagnostics): no live proxied traffic ran in this workflow, so no spans were exported/collected during this run — expected, not a failure.

No errors or unexpected failures found. Graceful degradation confirmed (module loads and functions without throwing when OTEL is unconfigured).

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python 3.12.14 3.12.14 ✅ YES
Node.js v24.19.0 v22.23.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall: FAILED — Node.js version differs between host and chroot environments. smoke-chroot label not applied.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor
  • fix: filter workflow commands from microVM output
  • feat: verify virtiofsd sandbox confinement before VM boot
  • GitHub PR review: ✅
  • safeinputs-gh PR query: ❌
  • Playwright title check: ✅
  • File write/read: ✅
  • Discussion query/comment: ❌
  • AWF build: ✅
  • Overall: FAIL

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • msfeed25.pkgs.visualstudio.com
  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "msfeed25.pkgs.visualstudio.com"
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 2698c6b into main Aug 31, 2026
166 of 175 checks passed
@lpcox
lpcox deleted the lpcox-signed-artifact-manifest branch August 31, 2026 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants