feat: cross-compile & upgrade LLVM to 21.1.8

[16bit-ykiko]

Summary

This PR adds cross-compilation support for three new target platforms, upgrades LLVM to 21.1.8, and overhauls the CI pipelines around cross-builds and testing.

Cross-compilation

New target triples accepted via -DCLICE_TARGET_TRIPLE=...:

Target triple	Host	Output
x86_64-apple-darwin	macos-15 (arm64)	macOS x64
aarch64-linux-gnu	ubuntu-24.04 (x64)	Linux arm64
aarch64-pc-windows-msvc	windows-2025 (x64)	Windows arm64

• cmake/toolchain.cmake — maps CLICE_TARGET_TRIPLE to CMAKE_SYSTEM_NAME/CMAKE_SYSTEM_PROCESSOR/compiler --target; picks up conda aarch64 sysroot when cross-compiling Linux.
• cmake/llvm.cmake — forwards target platform/arch to setup-llvm.py so the right prebuilt LLVM is downloaded for the target.
• CMakeLists.txt — uses a host-side flatc from PATH under CMAKE_CROSSCOMPILING instead of the in-tree target build.
• pixi.toml:
  • Adds osx-64, linux-aarch64, win-arm64 platforms.
  • New environments: cross-macos-x64, cross-linux-aarch64 (adds gcc_linux-aarch64 + sysroot_linux-aarch64), cross-windows-arm64.
  • New lightweight test-run env used on native ARM/x64 runners to execute cross-built artifacts (pulls in upstream clang+lld on macOS so tests don't fall back to Apple clang).
• scripts/activate_cross_linux.sh — exports CONDA_PREFIX-relative paths for the aarch64 toolchain.
• scripts/build-llvm.py — --target-triple support and a build_native_tools() helper that produces host llvm-tblgen / clang-tblgen needed when cross-compiling LLVM itself.

LLVM upgrade 21.1.4 → 21.1.8

• cmake/package.cmake bumps setup_llvm("21.1.8").
• config/llvm-manifest.json regenerated with 6 new cross-compiled entries and a new arch field on every entry so lookup is (version, platform, arch, lto, build_type).
• scripts/setup-llvm.py — honours the new arch field when resolving artifacts.
• scripts/update-llvm-version.py (new) — single-call version bump across package.cmake + manifest.
• scripts/validate-llvm-components.py (new) — scans the LLVM source tree for library targets and diffs them against scripts/llvm-components.json to catch stale/misspelled component names before a build.
• scripts/llvm-components.json (new) — explicit allow-list of required LLVM/Clang library targets used by build-llvm.py.

CI changes

• .github/workflows/build-llvm.yml:
  • Adds workflow_dispatch with llvm_version, skip_upload, skip_pr inputs.
  • Matrix extended with the 6 cross-compile entries (2 per new platform: RelWithDebInfo ± LTO).
  • build clice / test / prune steps gated on !matrix.target_triple for cross-builds; cross-built LTO entries apply the native prune manifest (arch-independent).
  • Cross-compiled binary architecture is verified with file(1).
  • New upload job triggered by workflow_dispatch pushes artifacts to clice-io/clice-llvm and hands the manifest off to the next job.
• .github/workflows/test-cmake.yml:
  • Build matrix gains three build_only: true cross entries that upload bin/ + lib/ artifacts.
  • New test-cross job runs on native macos-15-intel, ubuntu-24.04-arm, windows-11-arm runners, downloads the cross-built artifacts, and runs unit / integration / smoke tests under the test-run pixi env.
  • Cache keys now include target_triple so native and cross builds don't collide.
• .github/workflows/publish-clice.yml:
  • Three additional release artifacts for the new targets (clice-x86_64-macos-darwin, clice-aarch64-linux-gnu, clice-aarch64-windows-msvc), each with a matching -symbol archive.

Compatibility

• All existing native builds and tests are preserved; cross entries are additive.
• Debug + ASAN remains disabled on Windows (llvm_mode == Debug && os == windows-* no longer appends -asan).

🤖 Generated with Claude Code

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds cross-compilation support across CI and build tooling: new target triples in GitHub Actions, propagation of CLICE_TARGET_TRIPLE through scripts/CMake/toolchain, conditional CI steps for native vs cross builds, LLVM version bumped, and packaging/prune logic adjusted for cross targets.

Changes

Cohort / File(s)	Summary
CI Workflows \.github/workflows/build-llvm.yml, \.github/workflows/publish-clice.yml	Add workflow_dispatch; add cross-compilation matrix entries with target_triple; gate native-only vs cross steps; unify build to CMake/Ninja; change artifact naming/paths and prune/apply scoping; compute archive extension from matrix.asset_name.
CMake: LLVM & Package cmake/llvm.cmake, cmake/package.cmake	setup_llvm now accepts optional --target-platform when CLICE_TARGET_TRIPLE set; bumped LLVM setup version llvmorg-21.1.4 → llvmorg-21.1.8.
CMake: Toolchain cmake/toolchain.cmake	Add CLICE_TARGET_TRIPLE-driven cross-compilation config for x86_64-apple-darwin, aarch64-*-linux, aarch64-*-windows: sets CMAKE_SYSTEM_NAME/processor, compiler target, and sysroot handling.
Build Scripts scripts/build-llvm.py, scripts/setup-llvm.py	Add CLI flags --target-triple (propagated to CMake as -DCLICE_TARGET_TRIPLE) and --target-platform (override artifact/platform selection); build script stops forcing -DLLVM_USE_LINKER=lld.
Config & Dependencies config/llvm-manifest.json, pixi.toml	Clear llvm-manifest.json to []; add cross environments in pixi.toml including cross-linux-aarch64 with sysroot and cross gcc/g++ deps; two cross env placeholders added for macOS/Windows.
Packaging/Artifacts \.github/workflows/publish-clice.yml (asset naming)	Replace artifact_name/symbol_artifact_name with asset_name/symbol_asset_name; update upload paths to build/release/... and symbol naming to clice-symbol....

Sequence Diagram

sequenceDiagram
    participant User
    participant CI as CI Runner
    participant CMake
    participant Toolchain as cmake/toolchain.cmake
    participant SetupLLVM as setup-llvm.py
    participant Builder as Build (cmake/ninja)

    User->>CI: trigger workflow (matrix may set target_triple)
    CI->>CMake: run cmake -B build/release ... (-DCLICE_TARGET_TRIPLE=TRIPLE)?
    CMake->>Toolchain: read CLICE_TARGET_TRIPLE
    Toolchain->>Toolchain: configure system name / processor / compiler target / sysroot
    CMake->>SetupLLVM: setup_llvm(... --target-platform=PLATFORM?)
    SetupLLVM->>SetupLLVM: select artifact manifest for PLATFORM
    CMake->>Builder: cmake --build build/release
    Builder->>CI: produce build artifact
    CI->>CI: run native or cross prune/apply/upload steps (gated by matrix)

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• build: use pixi for managing build toolchains #322 — Modifies CI/tooling, pixi.toml and CMake/build scripts overlapping this PR’s changes.
• build: use pixi in ci #325 — Related CI and pixi configuration updates that touch cross-build flows.
• refactor(cmake): clean up toolchain and CMakeLists separation #379 — Touches cmake/toolchain.cmake and linker/compiler launcher logic; likely intersects with the new toolchain changes.

Suggested reviewers

• star-hengxing

Poem

🐰 Hopping through runners, I cheer and twirl,
Triples stacked like ribbons, flags unfurl,
macOS, Linux, Windows in a line,
LLVM bumped — the build logs shine,
A rabbit packs artifacts, ready to whirl.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the two main changes: adding cross-compilation support and upgrading LLVM to version 21.1.8, matching the core objectives of the PR.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/cross-compile

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

.github/workflows/build-llvm.yml (1)

171-216: ⚠️ Potential issue | 🟠 Major

Make the prune manifest producer a separate job.

The +LTO legs download llvm-pruned-libs-${{ matrix.os }} from the same github.run_id, but sibling matrix legs have no ordering guarantee. If a consumer leg reaches apply before the native RelWithDebInfo/OFF leg uploads its artifact, this becomes a flaky CI failure.

🧹 Nitpick comments (3)

config/llvm-manifest.json (1)

1-1: Empty manifest will block builds until regenerated.

The manifest is intentionally cleared for the LLVM 21.1.8 upgrade. However, until build-llvm.yml and upload-llvm.yml are run to regenerate artifacts, any build that doesn't have a pre-existing LLVM install will fail in pick_artifact() with "No matching LLVM artifact in manifest".

Ensure the follow-up actions (run build-llvm.yml → upload-llvm.yml) are executed promptly after merging to restore build functionality.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@config/llvm-manifest.json` at line 1, The LLVM manifest file was cleared for
the 21.1.8 upgrade which leaves pick_artifact() unable to find artifacts and
will block builds; after merging, immediately run the build workflow
(build-llvm.yml) to produce artifacts and then run the upload workflow
(upload-llvm.yml) to update config/llvm-manifest.json so pick_artifact() can
locate matches; verify the manifest is populated and that pick_artifact()
succeeds in CI before closing the change.

.github/workflows/build-llvm.yml (2)

226-235: Fail fast on unsupported target triples.

Right now a typo or future triple addition leaves ARCH/PLATFORM/TOOLCHAIN unset and still produces a malformed artifact name.

Suggested hardening

           if [[ -n "${{ matrix.target_triple }}" ]]; then
             case "${{ matrix.target_triple }}" in
               x86_64-apple-darwin)
                 ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;;
               aarch64-linux-gnu)
                 ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;;
               aarch64-pc-windows-msvc)
                 ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;;
+              *)
+                echo "Unsupported target triple: ${{ matrix.target_triple }}" >&2
+                exit 1 ;;
             esac
           else

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.github/workflows/build-llvm.yml around lines 226 - 235, The case block that
sets ARCH/PLATFORM/TOOLCHAIN for matrix.target_triple can leave those variables
unset for typos or unknown triples; add a default/fallback case in the case
statement (for the block handling "${{ matrix.target_triple }}") that logs an
explicit error mentioning the unrecognized target triple and exits non‑zero, and
also after the case ensure a guard that checks ARCH/PLATFORM/TOOLCHAIN are set
(e.g., test -n "$ARCH" && ...) to fail fast; update references to
ARCH/PLATFORM/TOOLCHAIN in the surrounding script to rely on this explicit
failure rather than producing malformed artifact names.

---

198-216: Please validate cross-target pruning before packaging.

These legs apply a manifest learned from the native arch and then upload immediately, but they never build a cross-target consumer against the pruned install. If the required LLVM library closure differs by target triple, this workflow can publish a broken archive without noticing. A per-target manifest or a small post-prune link smoke test would make this much safer.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.github/workflows/build-llvm.yml around lines 198 - 216, The workflow
applies a native ARCH manifest using the step named "Apply pruned-libs manifest
(cross-compile + LTO)" (which calls scripts/prune-llvm-bin.py with
MANIFEST="pruned-libs-${{ matrix.os }}.json") but never verifies the pruned
install for the cross-target; add a validation step after running
prune-llvm-bin.py and before uploading the GH artifact "llvm-pruned-libs-${{
matrix.os }}" that either (a) generates a per-target manifest by running the
pruning script on the target toolchain/headers, or (b) performs a small
smoke/link test that compiles and links a minimal program against the pruned
install at .llvm/build-install/lib (using the matrix.target_triple and matrix.os
environment) to ensure all required libraries are present; implement the smoke
test as a new step that returns non-zero on failure so the job fails before
publishing the artifact.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pixi.toml`:
- Line 43: The change added a new workspace constraint sysroot_linux-aarch64 =
"==2.17" in pixi.toml but the pixi.lock file is now out of sync; regenerate the
lock and commit it by running `pixi lock` locally (or in CI), verify the updated
pixi.lock contains the new sysroot_linux-aarch64 entry, and include the updated
pixi.lock in the commit so the pipeline no longer fails.

---

Nitpick comments:
In @.github/workflows/build-llvm.yml:
- Around line 226-235: The case block that sets ARCH/PLATFORM/TOOLCHAIN for
matrix.target_triple can leave those variables unset for typos or unknown
triples; add a default/fallback case in the case statement (for the block
handling "${{ matrix.target_triple }}") that logs an explicit error mentioning
the unrecognized target triple and exits non‑zero, and also after the case
ensure a guard that checks ARCH/PLATFORM/TOOLCHAIN are set (e.g., test -n
"$ARCH" && ...) to fail fast; update references to ARCH/PLATFORM/TOOLCHAIN in
the surrounding script to rely on this explicit failure rather than producing
malformed artifact names.
- Around line 198-216: The workflow applies a native ARCH manifest using the
step named "Apply pruned-libs manifest (cross-compile + LTO)" (which calls
scripts/prune-llvm-bin.py with MANIFEST="pruned-libs-${{ matrix.os }}.json") but
never verifies the pruned install for the cross-target; add a validation step
after running prune-llvm-bin.py and before uploading the GH artifact
"llvm-pruned-libs-${{ matrix.os }}" that either (a) generates a per-target
manifest by running the pruning script on the target toolchain/headers, or (b)
performs a small smoke/link test that compiles and links a minimal program
against the pruned install at .llvm/build-install/lib (using the
matrix.target_triple and matrix.os environment) to ensure all required libraries
are present; implement the smoke test as a new step that returns non-zero on
failure so the job fails before publishing the artifact.

In `@config/llvm-manifest.json`:
- Line 1: The LLVM manifest file was cleared for the 21.1.8 upgrade which leaves
pick_artifact() unable to find artifacts and will block builds; after merging,
immediately run the build workflow (build-llvm.yml) to produce artifacts and
then run the upload workflow (upload-llvm.yml) to update
config/llvm-manifest.json so pick_artifact() can locate matches; verify the
manifest is populated and that pick_artifact() succeeds in CI before closing the
change.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fbdf875e-89a4-43b2-98a1-39a2520a9284

📥 Commits

Reviewing files that changed from the base of the PR and between 1dd94e5 and 13cb0b1.

📒 Files selected for processing (9)

• .github/workflows/build-llvm.yml
• .github/workflows/publish-clice.yml
• cmake/llvm.cmake
• cmake/package.cmake
• cmake/toolchain.cmake
• config/llvm-manifest.json
• pixi.toml
• scripts/build-llvm.py
• scripts/setup-llvm.py

[coderabbitai]

🧹 Nitpick comments (1)

.github/workflows/build-llvm.yml (1)

224-246: Consider adding a fallback case for unknown target triples.

The case statement handles the three expected target triples but lacks a default case. If a new target triple is added to the matrix without updating this packaging block, ARCH, PLATFORM, and TOOLCHAIN would remain unset, causing a confusing packaging failure downstream.

🛡️ Proposed fix to add a fallback case

           case "${{ matrix.target_triple }}" in
             x86_64-apple-darwin)
               ARCH="x64"; PLATFORM="macos"; TOOLCHAIN="clang" ;;
             aarch64-linux-gnu)
               ARCH="aarch64"; PLATFORM="linux"; TOOLCHAIN="gnu" ;;
             aarch64-pc-windows-msvc)
               ARCH="aarch64"; PLATFORM="windows"; TOOLCHAIN="msvc" ;;
+            *)
+              echo "Error: Unknown target triple '${{ matrix.target_triple }}'"
+              exit 1 ;;
           esac

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In @.github/workflows/build-llvm.yml around lines 224 - 246, The case handling
for "${{ matrix.target_triple }}" can leave ARCH/PLATFORM/TOOLCHAIN unset for
unknown triples; add a fallback branch (the *) in that case statement to either
set sensible defaults (e.g., ARCH="x64", PLATFORM="linux", TOOLCHAIN="gnu") or
emit a clear error and exit, and include a warning message that shows the
unrecognized "${{ matrix.target_triple }}" so downstream packaging doesn't run
with empty variables.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/build-llvm.yml:
- Around line 224-246: The case handling for "${{ matrix.target_triple }}" can
leave ARCH/PLATFORM/TOOLCHAIN unset for unknown triples; add a fallback branch
(the *) in that case statement to either set sensible defaults (e.g.,
ARCH="x64", PLATFORM="linux", TOOLCHAIN="gnu") or emit a clear error and exit,
and include a warning message that shows the unrecognized "${{
matrix.target_triple }}" so downstream packaging doesn't run with empty
variables.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 450a2b15-6fee-4107-bda9-0c147c4c5182

📥 Commits

Reviewing files that changed from the base of the PR and between d66e891 and 300a5aa.

📒 Files selected for processing (1)

• .github/workflows/build-llvm.yml