Skip to content

Commit f3e3b5e

Browse files
committed
[anneal][v2] Add and integrate nix-built exocrate
- Add nix derivations for exocrate - Add `setup` sub-command to locate-or-install exocrate - Add integration test for "developer mode" `setup` that presumes local exocrate archive - Add github workflows to: - Warm nix cache - Locally link exocrate archive from nix, then run all tests gherrit-pr-id: Gwhroikc5idscowxamayknlke2uiddzv3
1 parent 4f657b1 commit f3e3b5e

909 files changed

Lines changed: 390095 additions & 4 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/anneal.yml

Lines changed: 89 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -274,7 +274,7 @@ jobs:
274274
duration=$((end - start))
275275
echo "Docker Pull Time: $duration seconds"
276276
echo "[{\"name\": \"Docker Pull Time\", \"unit\": \"seconds\", \"value\": $duration}]" > pull_time.json
277-
277+
278278
docker tag ghcr.io/google/zerocopy/anneal:${STEPS_DOCKER_TAG_OUTPUTS_TAG} anneal-ci:local
279279
env:
280280
STEPS_DOCKER_TAG_OUTPUTS_TAG: ${{ steps.docker_tag.outputs.tag }}
@@ -455,6 +455,93 @@ jobs:
455455
fi
456456
fi
457457
458+
v2_nix_cache:
459+
name: Warm Nix Cache for V2
460+
runs-on: ubuntu-latest
461+
needs: build_docker_env
462+
permissions:
463+
contents: read
464+
id-token: write # Required to exchange GitHub OIDC tokens for Determinate Systems Cache API access
465+
steps:
466+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
467+
with:
468+
persist-credentials: false
469+
470+
- name: Install Nix
471+
uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff # v3.6.1
472+
473+
- name: Run Magic Nix Cache
474+
uses: DeterminateSystems/magic-nix-cache-action@main # zizmor: ignore[unpinned-uses]
475+
476+
# On Ubuntu 24.04 (currently `ubuntu-latest`), AppArmor restricts unprivileged user namespaces by default.
477+
# The Nix build sandbox runs `steam-run` (which uses `bubblewrap`/`bwrap`) during the `mathlib-cache-download`
478+
# phase to create an FHS environment. `bwrap` requires creating a user namespace to set up uid mappings,
479+
# which fails with "Permission denied" unless this restriction is temporarily disabled on the host.
480+
#
481+
# We temporarily disable it right before the `nix build` step and re-enable it immediately after
482+
# to maintain the principle of least privilege.
483+
- name: Enable unprivileged user namespaces (Ubuntu 24.04)
484+
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
485+
486+
- name: Warm Nix Cache
487+
run: nix build .#omnibus-archive
488+
working-directory: anneal/v2
489+
490+
# Re-enable the AppArmor namespace restriction to restore the runner host's default security posture.
491+
# `if: always()` ensures this cleanup step runs even if the Nix build fails.
492+
- name: Restore AppArmor restriction
493+
if: always()
494+
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=1
495+
496+
v2:
497+
name: Run V2 tests
498+
runs-on: ubuntu-latest
499+
# Depending on `v2_nix_cache` avoids duplicate work and ensure `nix build ...` step for this job is fast.
500+
needs: [build_docker_env, v2_nix_cache]
501+
permissions:
502+
contents: read
503+
id-token: write # Required to exchange GitHub OIDC tokens for Determinate Systems Cache API access
504+
steps:
505+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
506+
with:
507+
persist-credentials: false
508+
509+
- name: Install Nix
510+
uses: DeterminateSystems/determinate-nix-action@441b9e401ac050c38a07d8313748c5c2d17e8aff # v3.6.1
511+
512+
- name: Run Magic Nix Cache
513+
uses: DeterminateSystems/magic-nix-cache-action@main # zizmor: ignore[unpinned-uses]
514+
515+
# On Ubuntu 24.04 (currently `ubuntu-latest`), AppArmor restricts unprivileged user namespaces by default.
516+
# The Nix build sandbox runs `steam-run` (which uses `bubblewrap`/`bwrap`) during the `mathlib-cache-download`
517+
# phase to create an FHS environment. `bwrap` requires creating a user namespace to set up uid mappings,
518+
# which fails with "Permission denied" unless this restriction is temporarily disabled on the host.
519+
#
520+
# We temporarily disable it right before the `nix build` step and re-enable it immediately after
521+
# to maintain the principle of least privilege.
522+
- name: Enable unprivileged user namespaces (Ubuntu 24.04)
523+
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
524+
525+
- name: Build outside-cargo dependencies (cached)
526+
run: |
527+
mkdir -p target
528+
nix build .#omnibus-archive --out-link target/anneal-exocrate.tar.zst
529+
working-directory: anneal/v2
530+
531+
# Re-enable the AppArmor namespace restriction to restore the runner host's default security posture.
532+
# `if: always()` ensures this cleanup step runs even if the Nix build fails.
533+
- name: Restore AppArmor restriction
534+
if: always()
535+
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=1
536+
537+
- name: Install latest nightly Rust
538+
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # zizmor: ignore[superfluous-actions]
539+
with:
540+
toolchain: nightly
541+
542+
- name: Run V2 tests
543+
run: cargo test --workspace --all-features # include, e.g., tests that assume exocrate prebuilt
544+
working-directory: anneal/v2
458545

459546
# Used to signal to branch protections that all other jobs have succeeded.
460547
all-jobs-succeed:
@@ -469,7 +556,7 @@ jobs:
469556
# https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/collaborating-on-repositories-with-code-quality-features/troubleshooting-required-status-checks#handling-skipped-but-required-checks
470557
if: failure()
471558
runs-on: ubuntu-latest
472-
needs: [build_docker_env, anneal_tests, verify_examples, measure_image_size]
559+
needs: [build_docker_env, anneal_tests, verify_examples, measure_image_size, v2_nix_cache, v2]
473560
steps:
474561
- name: Mark the job as failed
475562
run: exit 1

anneal/src/main.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ mod util;
1414

1515
use clap::Parser;
1616

17-
/// Anneal: A Literate Verification Toolchain
17+
/// Anneal
1818
#[derive(Parser, Debug)]
1919
#[command(name = "cargo-anneal", version, about, long_about = None)]
2020
struct Cli {

0 commit comments

Comments
 (0)