Add stable-haskell GHC (ghc914-sh): GHC 9.14 built with cabalProject instead of hadrian; darwin→linux cross via hyper-linux - #2528
Open
hamishmack wants to merge 74 commits into
Open
Add stable-haskell GHC (ghc914-sh): GHC 9.14 built with cabalProject instead of hadrian; darwin→linux cross via hyper-linux#2528hamishmack wants to merge 74 commits into
ghc914-sh): GHC 9.14 built with cabalProject instead of hadrian; darwin→linux cross via hyper-linux#2528hamishmack wants to merge 74 commits into
Conversation
plan-to-nix capitalizes cabal arch names (ppc → isPpc), but nixpkgs spells several predicates differently (isPPC, isRiscV64, isLoongArch64). Map them in fixSystem and keep aliases in cabal-os-arch-comp.nix so already-generated plan.nix files continue to evaluate. Covers the architectures referenced by GHC 9.14's rts.cabal.
Allows pre-built Haskell packages to be made available as dependencies of individual components (their .conf files are added to the component's package DB), merging with the project-wide prebuilt-depends. Config-files derivations track regularLibDeps separately so prebuilt-depends do not leak into the setup package DBs of downstream packages. Used by the sghc914 (stable-haskell GHC) build, where GHC 9.14's ghc-internal.conf depends on all rts sub-libraries.
Linux: c, rt, dl (provided by the toolchain). Windows: wsock32, dbghelp, psapi (rts.cabal) and ntdll, ucrt, mingw32 (ghc-internal.cabal) — all provided implicitly by the mingw toolchain.
GHC 9.14's rts.conf depends on libffi-clib at the ghc-pkg level. Cabal does not track it in the plan (it is a C library wrapper), so add it to nonReinstallablePkgs so make-config-files copies its .conf into per-component package DBs. Harmless for compilers without it.
A package whose only plan target is disabled (e.g. unix on Windows in a --disable-tests project) previously crashed evaluation in defaultTargetId ((builtins.head available).id on the reason string: "expected a set but found a string"). Filter it like TargetNotBuildable.
Compilers that are not built with hadrian (e.g. the cabalProject-built sghc914) have no hadrianProject passthru and no iserv-proxy-exes entry; roots' previously assumed both exist.
New cabal-project option (default false): when enabled, direct Hackage tarball URLs in the cabal.project packages: stanza (https://hackage.haskell.org/package/NAME-VER/NAME-VER.tar.gz) are rewritten to local nix store paths fetched via hackage.nix, so plan-to-nix needs no network and the pinned versions need not be present at the project index-state. - overlays/haskell.nix: add reusable haskell-nix.hackageTarball helper (fetch+unpack an exact version using the for-stackage db sha256). - modules/cabal-project.nix: declare replace-hackage-tarball-urls. - lib/call-cabal-project-to-nix.nix: rewrite rawCabalProject via replaceStrings (preserving store-path context). Used by the stable-haskell GHC (sghc914) build, whose upstream cabal.project pins its boot libraries this way.
Adds haskell-nix.compiler.sghc914 (and sghc914-stage1), a GHC built from
the stable-haskell/ghc stable-ghc-9.14 branch using haskell.nix's own
cabalProject machinery instead of hadrian, following the branch's
Makefile / cabal.project.* files:
stage1 - GHC executables built with the boot compiler (ghc9103)
stage2 - all boot libraries and executables built with stage1
cross - target boot libraries built with a cross-targeting stage1;
the final cross compiler is native stage1 binaries + target
settings (ghc-toolchain) + the target package DB
Every cabal component is its own derivation; the compiler closure is
2.4 GiB vs 3.9 GiB for the hadrian-built ghc9141 on aarch64-darwin.
Source pins live in lazy-inputs/ (sghc914 plus the sghc914-cabal /
sghc914-hpc-bin / sghc914-hsc2hs source-repository-packages, wired via
inputMap). The overlay also provides:
- dumpSrc: a raw-src view that completes the plan-time installed-package
dump with the hackage-URL-pinned boot libraries (no longer in-tree)
- wasm: libffi-wasm built and installed alongside the rts headers
- ghcjs: emscripten toolchain, JS-backend rts-way flags
- Windows: version-aware boot-lib registration (cabal legitimately
solves a second older universe inside the Win32:hsc2hs:exe scope to
break the Win32 -> hsc2hs -> process -> Win32 build-tool cycle) and
cross-host compiler registration so templateHaskell/iserv-proxy-exes
include sghc914 (with the TH-wrapper recursion cut in crossBootProject)
- dev shells (haskell-nix.sghc914-shells): from-boot and from-stage1,
for hacking on the GHC tree with cabal
Verified: native aarch64-darwin and wasi32 build and run; all other
hydraJobs targets (ghcjs, musl32/64, static, aarch64-multiplatform(-musl),
android, mingwW64, ucrt64, x86_64-linux) instantiate.
hydraJobs.<system>.unstable.sghc914.native.hello verified building and running on aarch64-darwin; wasi32.hello runs under wasmtime.
useLocalGhcLib / ghc-lib-reinstallable and ghc-boot-packages consume ghc.generated during evaluation: plan-to-nix realises the GHC-tree source-repository-package to compute the v2 UnitId. Because 'generated' was an output of the full GHC derivation, that forced a from-source compiler build via IFD whenever it wasn't cached -- the cause of the multi-hour / hung hydra evaluations on this branch. Add passthru.generated-light: a lightweight derivation that runs only hadrian's generated-source targets (genprimopcode / deriveConstants / template substitution, all stage0Boot) instead of compiling all of GHC. The primop-*.hs-incl set is discovered per-version from hadrian's Rules/Generate.hs; targets are built with -j1 to avoid a genprimopcode scheduling race. Also fixes a pre-existing .hss typo so ghc-boot's Version.hs is included. Route consumers (ghc-packages.nix, cabal-project.nix, stack-project.nix, ghcjs-project.nix) through 'generated-light or generated', and cache it via roots'. sghc914 has neither attr and is unaffected (it builds lib:ghc from configuredSrc with build-time generation). Add test/generated-light comparing generated-light against the full-build generated byte-for-byte.
…D_RTS in Cmm) Root cause: the stable-haskell fork's threaded RTS compiled its .cmm with THREADED_RTS undefined (cc-options reached the C sources but none of the four stanza forms lands -optc-DTHREADED_RTS on the .cmm compile). So StgMiscClosures.cmm's stg_WHITEHOLE_info took the non-threaded barf branch inside the threaded RTS, making any transient WHITEHOLE lock under 'ghc --make -jN' fatal on aarch64-darwin. Fixed at the source (stable-haskell/ghc#191, hkm/rts-threaded-cmm-optc): rts-threaded-flags now also passes cmm-options: -optc-DTHREADED_RTS. Repin sghc914 to that branch. Verified: stg_WHITEHOLE_info spins, 120-module -j8 repro 0/N on both the debug and non-debug threaded RTS. overlays/stable-haskell.nix also forces 'package ghc-bin flags: -debug' so the compiler links the non-debug rts_thr (matching hadrian; the fork's ghc-bin debug flag defaults True). This is release-shaping only, orthogonal to the crash. Also carries in-flight branch edits: head.hackage tarball hash bump (test/cabal.project.local), boot-package guard (overlays/ghc-packages.nix), and CLAUDE.md notes.
The fork's cabal.project.common disables both (package text flags: -simdutf,
package haskeline flags: -terminfo). Current haskell.nix can build them:
terminfo's ncurses is auto-wired (--extra-{include,lib}-dirs to ncurses-6.6,
setupterm found in -ltinfo) and text's simd path links system-cxx-std-lib.
Override the flags to +simdutf/+terminfo in the overlay cabalProjectLocal, and
register terminfo + system-cxx-std-lib in the final compiler's package DB
(bootLibraries) — without that, text/haskeline are broken by dangling deps
('unusable due to missing dependencies').
Verified: ghc-pkg check clean (no broken packages), a Data.Text UTF-8 roundtrip
program compiles/links/runs, ghci loads, and the 120-module -j8 repro stays 0/N.
Mechanical rename of the stable-haskell GHC 9.14 compiler and its lazy-inputs:
- compiler attr haskell-nix.compiler.sghc914 -> ghc914-sh
- lazy-inputs/{sghc914,sghc914-cabal,sghc914-hpc-bin,sghc914-hsc2hs} ->
ghc914-sh{,-cabal,-hpc-bin,-hsc2hs} (dirs + flake input names + locks)
- haskell-nix.sources.sghc914* -> ghc914-sh*
- derivation names, internal bindings, ci.nix, CLAUDE.md, generated-light note
Content unchanged; eval resolves (compiler.ghc914-sh.drvPath, sources.ghc914-sh).
stable-haskell/ghc#191 merged into stable-ghc-9.14 (tip 2c2ebfa; tree identical to the fix branch 0fee7fb). Point the pin back at the mainline branch and drop the temporary hkm/rts-threaded-cmm-optc reference. narHash uses the flake-compat FOD value (nix flake update's differs — known gotcha).
The dummy ghc-pkg dump that cabal's solver sees as the installed package set is synthesised from .cabal files found in the compiler's raw-src. Two classes of ghc914-sh boot libraries were missing from it: - Cabal and Cabal-syntax come from a source-repository-package (the stable-haskell/Cabal fork), so they are neither in the GHC tree nor covered by the hackage-URL overlay in dumpSrc. The dump then omitted them entirely and silently dropped ghc-boot's Cabal-syntax depends edge, so user-project plans (e.g. iserv-proxy for cross TH) never marked Cabal-syntax pre-existing and the pruned per-component package DB failed with 'installed package ghc-boot-9.14 is broken due to missing package Cabal-syntax'. Overlay their .cabal files into dumpSrc from the ghc914-sh-cabal lazy-input. - text is built with +simdutf in stage2, so the installed text depends on system-cxx-std-lib, but that build-depends sits behind a flag conditional the dump extraction cannot see. Add the edge in call-cabal-project-to-nix.nix gated on a new compiler passthru enableTextSimdutf (mirroring the haskeline/terminfo special case). Only the native ghc914-sh sets it: the cross boot libs build text WITHOUT simdutf.
isNativeMusl only compared linuxArch across build/host/target, but
linuxArch is defined for any OS (aarch64-darwin is "arm64" too), so an
aarch64-darwin -> aarch64-linux-musl cross was treated as native musl.
That enabled ghc-for-component-wrapper.nix's unprefixed unlit/ghc-iserv
alias block, which self-collides for compilers with an empty
targetPrefix ('ln: failed to create symbolic link bin/unlit: File
exists' while building libffi-clib), and would generally let musl
target binaries be treated as runnable on the build machine.
Require buildPlatform.isLinux; on Linux build hosts this is always
true, so existing behaviour is unchanged.
On Linux build hosts, cross target binaries (TH via iserv, tests) run under qemu user-mode emulation, which does not exist on macOS. hyper-linux (github:zw3rk/hyper-linux) runs aarch64-linux and x86_64-linux ELF binaries directly on Apple Silicon via Hypervisor.framework, and slots into the same emulator seat: - overlays/linux-cross.nix: the emulator invocation is now a pipeCommand argument defaulting to the previous qemu user-mode command (when overridden, the qemu argument is never forced, so darwin hosts do not build qemu). Also fix a $@* typo in the test wrapper. - overlays/armv6l-linux.nix: darwin build hosts pass pipeCommand = pkgsBuildBuild.hyper-linux + /bin/hl (with a clear throw when the hyper-linux overlay attribute is not provided; there is no in-repo pin yet). Extend isLinuxCross to x86_64 targets: those are only a cross HOST from a non-Linux or non-x86_64 build platform (x86_64-linux -> musl64 stays native musl / isCrossHost false, so Linux hosts are unaffected). - lib/default.nix: qemuByHostPlatform learns x86_64. Verified end-to-end on aarch64-darwin with ghc914-sh -> aarch64-unknown-linux-musl: hackage hello and a Template Haskell example (test/th-dlls-minimal, TH evaluated by iserv-proxy piped through hl into the static musl iserv-proxy-interpreter) build and run under hl. Known issues (upstream hl 0.2.4): hl SIGBUSes when spawned inside a nix-daemon build, so in-sandbox TH currently fails (works when the same derivation's phases run as a login user); x86_64 guests need a 48-bit IPA VM for Rosetta and fail on M1 (M2+ only).
hamishmack
changed the title
sghc914): GHC 9.14 built with cabalProject instead of hadrianghc914-sh): GHC 9.14 built with cabalProject instead of hadrian
hadrian configures the stage0 (build platform) in-tree libraries with the same TARGET --with-ffi-libraries directory as stage1 (the same stage-conflation bug class as the --with-curses-libraries one noted at enableTerminfo). On Linux build hosts the same-arch ELF libffi happens to satisfy the stage0 link, but when cross-compiling from darwin, ld64 finds the ELF libffi.so, ignores it as wrong-format and STOPS searching (verified: later -L directories are not consulted for that -l), leaving ffi_call & co undefined when linking stage0 executables (e.g. haddock) against the in-tree rts/ghci. Fix: for cross compilers built on darwin, pass --with-ffi-libraries a symlinkJoin of the target libffi lib dir and the build-platform libffi. Each linker then sees only its own format: ld64 prefers libffi.dylib, while the target's GNU ld only ever looks for libffi.so/libffi.a. Linux builds keep the exact previous path, so no existing derivation hashes change. With this, ghc9141 (hadrian) builds as an aarch64-darwin -> aarch64-unknown-linux-musl cross compiler and its output (including a Template Haskell example evaluated via iserv-proxy piped through hyper-linux) runs correctly under hl.
Make the v2 slice builder reproduce plan-nix's UnitIds exactly when the compiler is a stable-haskell cabalProject GHC (ghc914-sh), so the leksah dev shell builds end-to-end (ghc 9.14 + cabal 3.16.1 + HLS 2.14). - stable-haskell.nix: unify every nix-facing surface on ghcVersion="9.14" (munged) — passthru.version, lib/ghc-9.14, bin aliases; ghcVersionFull "9.14.0" survives only as the stage1 ./configure --with-project-version arg. The compiler id is a pkgHash input, so 9.14.0 vs 9.14 forked it. Add -B$out/lib/ghc-9.14 to the assembled native stage2 ghc wrapper so topdir resolves unlit (GHC honours the last -B, so the component wrapper's -B still wins). New passthru projectUnitId="ghc-9.14-inplace". stage2Project module gives boot libs deterministic ids via --ipid=$pkgid[-inplace] (rts + sublibs and system-cxx-std-lib bare), matching what the stage2 plan and hadrian bindists register. - dummy-ghc.nix: branch on ghc.isStableHaskell so plan-time --info matches the real compiler — GHC Dynamic NO and RTS ways "v thr debug thr_debug" (statically-linked ghc-bin, no dyn/prof ways); Project Unit Id from projectUnitId. Cabal keys --enable-shared/--disable-shared and dep unit-ids on these, so a mismatch forks every downstream UnitId. - comp-v2-builder.nix: skip the slice unit-id check for style=="inplace" as well as "local". Under ghc914-sh the -inplace boot libs make cabal elaborate the generated haskell-gi gi-* family BuildInplaceOnly (fixed <pkgid>-inplace id) even though they're plain hackage deps; the slice serves identical source and builds them global-hashed, so the ids differ by construction. Also compose source setup-deps' slices and pin the transitive setup-dep closure as ownSetupConstraints so setups solve against the same deps plan-nix recorded. - comp-builder.nix: widen the library installCheck conf glob to also accept the bare <pkgid>.conf that an explicit --ipid registers, gated on components that actually pass --ipid (keeps other derivation hashes). - test/cabal.project.local: bump the rolling head.hackage --sha256.
hamishmack
changed the title
ghc914-sh): GHC 9.14 built with cabalProject instead of hadrianghc914-sh): GHC 9.14 built with cabalProject instead of hadrian; darwin→linux cross via hyper-linux
…ia hyper-linux Cross-compile Linux (musl) targets from an aarch64-darwin build host, running the target binaries under hyper-linux (`hl`) on Apple Silicon — qemu user-mode emulation doesn't exist on macOS. - Add a `hyper-linux` overlay (pinned `hyper-linux` flake input, nixpkgs follows ours) exposing `hl` as `pkgs.hyper-linux`, so darwin -> linux cross works out of the box instead of each consumer supplying it. `overlays/ armv6l-linux.nix` picks it up as the `pipeCommand` on darwin build hosts. Only defined where hyper-linux ships a `default` package (aarch64-darwin); lazy and absent elsewhere. - ci.nix: enable the `aarch64-multiplatform-musl` and `musl64` cross targets for unstable ghc9141 (hadrian) and ghc914-sh (stable-haskell) on aarch64-darwin. Build the compile side only — the cross `tests` are gated off for darwin -> linux because building their Template Haskell would run `hl` inside the nix build, which currently hits an `hl` SIGBUS under the nix-daemon; the compile side (roots + ghc + iserv-proxy-exes + hello) instantiates cleanly for all three tested combinations. - scripts/update-ci-status.sh: add `ghc914-sh` to COMPILERS (also realigns the ci-status-matrix guard test with the current hydraJobs) and the two new aarch64-darwin cross rows to PLATFORMS, so the README CI-status table tracks them.
Apple's ranlib (which cabal runs after `ar` on darwin build hosts) rewrites archives with 8-byte '\n' member padding counted into the header size field. GHC's archive reader then hands padded wasm cbits objects to emscripten and wasm-ld fails with "section too large". Patch GHC.SysTools.Ar to recover a wasm member's true length from its section table and drop the padding. Applied for GHC 9.12-9.15 on the JS backend.
head.hackage is a rolling repository, so the FOD that runs cabal v2-update against it goes stale periodically. Refresh the pinned hash.
Pull cabal-install/Cabal/Cabal-syntax/cabal-install-solver 3.17 from the stable-haskell/Cabal fork (the cross "stage" system, haskell/cabal#11179) via source-repository-package in nix-tools/cabal.project, so make-install-plan gets `with-build-compiler` and build:/host: stage-qualified constraints. Port nix-tools' sources to the Cabal-syntax 3.17 API (dropped the constraint type param from CondTree/CondBranch; CondNode lost its constraints field; *Sources became [ExtraSource Pkg]; Verbosity split into Verbosity/VerbosityFlags; plan values are Stage-tagged): - lib-cabal2nix/Cabal2Nix.hs - make-install-plan/{Freeze,MakeInstallPlan,ProjectPlanOutput}.hs (ProjectPlanOutput strips the build:/host: stage qualifier from every emitted dependency id so plan.json ids stay bare) - setup-ghcjs/Setup.hs Reconcile the installed-package-id-os-override patch to 3.17 (preserving the fork's OSX->Long unit-id format), and add a patch porting Cabal-syntax-json (the cabal2json provider) to the 3.17 CondTree/ExtraSource/Verbosity API.
v2CabalInstall now builds cabal-install 3.17.0.1 from the fork via source-repository-package (cabalProjectLocal), so the per-slice `cabal v2-build` has the stage system (--with-build-compiler / build:/host:). Reconcile the v2 cabal-install patches to the fork's 3.17 source: regenerate skip-installed-revdeps + setup-build-num-jobs-env at fuzz 0. The two prune-unreachable-sublibs patches are temporarily dropped (kept in-tree for reference) — their reachableSubLibs/reachablePDSubLibs walk `CondNode _ ds _` for the per-node [Dependency] constraints, which 3.17 removed from CondNode, so they need a data-model rewrite rather than an arity fix. They only affect sublib-heavy hackage packages (e.g. vector's benchmarks-O2), not the cabal tool or the ghc914-sh boot libs.
…fork
Drive plan-to-nix with the fork-linked nix-tools instead of the pinned static
tarball:
- overlays/default.nix: nix-tools-unchecked builds from source, bootstrapped
through the pinned haskell.nix (its own static nix-tools breaks the
plan-to-nix recursion — make-install-plan AND the hardcoded cabal2json both
consume nix-tools-unchecked); default-setup kept from the static pin.
- lib/call-cabal-project-to-nix.nix: add a plan-time `gcc` shim (execs the
build cc) for the fork's toolchain probe — UnitId-safe, cc isn't hashed;
add a `withBuildCompiler` path that builds a dummy build-platform ghc/ghc-pkg
(reporting the build boot libs installed) and passes
--with-build-compiler/--with-build-hc-pkg to make-install-plan, so the stage
solver is satisfied without building a real compiler.
- modules/cabal-project.nix: add the `withBuildCompiler` option.
- modules/hackage-project.nix: emit `optional-packages: ./*` in the generated
project so the fork's stricter parser accepts hackage-tool projects
(Cabal-7168) — UnitId-neutral, matches nothing locally.
Also includes the prepopulateHackageIndex option (minimal empty index for
:none projects) wired through call-cabal-project-to-nix, modules/cabal-project,
and overlays/haskell.nix's dotCabal.
The v2 builder inferred `documentation: True` for any unit whose plan-nix configure-args carry `--ghc-option=-haddock`. The marker is ambiguous: a plain `ghc-options: -haddock` (e.g. haskell-language- server's `package *` stanza) leaves the identical arg WITHOUT setting the haddock booleans that enter the UnitId hash (pkgHashDocumentation et al.). Misinferring forked every such slice's UnitId from plan-nix and made `cabal v2-build` run the haddock program, which stable-haskell compilers don't ship. plan.json offers no truthful per-unit signal (`haddock-args` is always the bare verb and the configure-args are byte-identical in both cases — verified against a `documentation: True` plan), so gate the inference on the assembled cabal.project + cabalProjectLocal text actually containing a `documentation: ... True` line. The text is threaded as a new pkg-set option `cabalProject` (mirroring `cabalProjectLocal`); when neither is available the old marker-only behaviour is kept. The eventual truthful fix is make-install-plan emitting `elabBuildHaddocks`, at the cost of a nix-tools (world-rebuild) change. Drv-neutral for projects that build today (verified: ghc914-sh native hello unchanged); with this, leksah's dev shell (HLS master as a `haskell-nix.tool` with ghc914-sh via v2) builds. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The -target cross wrapper runs ghc-toolchain-bin to generate the target settings. For the ghcjs target it aborted at "checking for 'ranlib'": ghc-toolchain probes PATH for javascript-unknown-ghcjs-ranlib / llvm-ranlib / ranlib, but emscripten ships emar / emranlib / emnm, so ar/ranlib/nm discovery (all fatal) never found them. Pass --ar/--ranlib/--nm explicitly for the ghcjs target, mirroring the AR/RANLIB/NM exports the hadrian build uses for GHC >= 9.12 (compiler/ghc/default.nix). The non-ghcjs branch is empty so the wrapper script stays byte-identical for every other target (musl64 hello drvPath unchanged). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
In a two-stage plan (stable-haskell `-target` cross compiler, whose target global package db — and hence the plan-time dummy `ghc-pkg dump` — is EMPTY) a boot library appears twice under one bare id: a pre-existing BUILD-stage unit (from the native build compiler) and the configured HOST-stage unit rebuilt from source (e.g. `base-4.22.0.0`). The dedup that normally drops such a shadow is intentionally skipped for two-stage plans. `by-id` is first-wins over the install-plan and the pre-existing entry comes first, so a HOST consumer's `depends` edge (`hello` → `base`) resolved to the pre-existing stub and `lookupDependency` dropped it as compiler-provided — even though the target db is empty. Every consumer of a boot lib (the user's exe, and the cross compiler's own Cabal / ghc-boot / ghci / iserv-proxy slices) then had an empty buildInputs/depSlices and cabal's solver failed with `next goal: host:base … fail (backjumping)`. Resolve host library/exe deps through a host-preferring id map (`by-id-host` = `by-id // configuredById`) and a matching `pre-existing-depends-host`, so the configured from-source twin wins and its slice is composed in; the whole boot closure then resolves from source. Setup/build-scope deps keep the pre-existing (build-stage, compiler-provided) boot lib via the unchanged `by-id`. Scoped to two-stage plans: for every other plan the host maps are defined to equal the build maps, so native ghc914-sh, hadrian, and ordinary projects are byte-identical (verified: ghc914-sh native hello `7ay62hzh` and hadrian ghc9141 hello `wijb55hj` drvPaths unchanged). With the fix both ghc914-sh musl64 and ghcjs hello now compose and build their from-source boot slices (remaining work: cross boot-lib unit-id alignment vs plan-nix, tracked on the branch). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The stable-haskell `-target` cross wrapper builds boot libraries from source but keeps `targetPrefix = ""` (it is the native compiler driven with `-target`), so `isCross` is false and the slice builder applied the strict "slice unit-id must equal plan-nix's" check to global-style (hashed) units such as libffi-clib. Those hashed ids are computed by cabal against the REAL cross wrapper, whose `ghc --info` differs from the plan-time dummy ghc — so they legitimately diverge from plan-nix's, exactly as on a genuine `isCross` build (where the check is already skipped). libffi-clib's slice built correctly but tripped the check (plan `65bbd76d…` vs slice `11a8cd48…`). Key the skip off the empty target global db (`emptyGlobalPackageDb`, the reliable two-stage-cross signal) in addition to `isCross`. Composition stays consistent: every consumer re-solves the unit from source against the same wrapper and agrees on the hashed id, and the target dummy dump is empty so no consumer reads these ids from a dump. Local/inplace units keep deterministic ids and are still checked. Drv-neutral for non-two-stage plans (ghc914-sh native hello 7ay62hzh, hadrian ghc9141 hello wijb55hj unchanged). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
GHC 9.14's unit wiring (mkUnitState) requires all four rts WAY sub-libraries (nonthreaded-nodebug, threaded-nodebug, …) to be resolvable whenever ghc-internal.conf is in a package's per-component db — which is every package that transitively links base. A native compiler satisfies this from its own (fully populated) global db, but a two-stage `-target` cross project has an EMPTY target global db and builds the boot libraries from source, so each HOST slice must compose the ways itself. Without them, `ghc-prim` (and every other boot lib) panics: "The RTS for rts:nonthreaded-nodebug is missing from the package database". Extend the boot-injection module (already gated on the empty target db via `shBootInject`) to set `additional-prebuilt-depends = rtsSublibs` on every HOST-stage package the plan configures — enumerated from plan-json (a fixed input; reading `config.packages` for the list would recurse through these very settings), excluding rts:nonthreaded-nodebug's own dependency closure (rts / libffi-clib / rts-fs / rts-headers) and BUILD-stage tools (alex / happy / … run natively with the build compiler, whose global db already has the ways). Mirrors the identical mechanism stage2Project uses for the native compiler build (overlays/stable-haskell.nix); the user's own packages need it too, not just the boot libs. `planUnitStage` is sourced from `pkgs.haskell-nix.haskellLib` (the file's own `haskellLib` module argument is declared but never provided by callers) and captured in the enclosing `let` so the nested pkg-set module closes over it rather than requesting it as a module arg. Drv-neutral for non-two-stage plans (ghc914-sh native hello 7ay62hzh, hadrian ghc9141 hello wijb55hj unchanged). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
directory, file-io, process and unix each carry an automatic `os-string` flag (default False) that selects — in both their .cabal build-depends and their .hsc/.hs sources — between the modern `os-string` package and filepath's pre-1.5 bundled `System.OsString.*` modules. With filepath-1.5.4.0 (which no longer bundles those modules) only the `+os-string` branch compiles. `allow-newer` relaxes the `else` branch's `filepath < 1.5.0.0` upper bound, so the solver otherwise keeps the default `-os-string` and pairs it with filepath-1.5.4.0 — a combination that builds a plan but fails at compile time with "hidden package os-string" / "Could not find module System.OsString.Data.ByteString.Short" (e.g. unix's System.Posix.Env.PosixString / System.Posix.PosixPath.FilePath). Force `+os-string` for these packages in the boot-injection cabalProjectLocal so plan-nix records the flag (and comp-v2-builder's flagBlockFor propagates it to each slice), matching how the compiler's own stage2 (cabal.project.stage2) resolves them. With the flag on, unix et al. now build from source under the -target cross wrapper. Drv-neutral for non-two-stage plans (block gated on the empty target db; ghc914-sh native hello 7ay62hzh, hadrian ghc9141 hello wijb55hj unchanged). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The stable-haskell cabal fork's `libraryConfigureInputsFromElabPackage` (cabal-install/src/Distribution/Client/InLibrary.hs) built the in-library LocalBuildInfo's `platform`/`compiler` from `getStage toolchains Host` unconditionally, while SetupWrapper builds the package's program db from `getStage toolchains (elabStage pkg)`. In a two-stage (cross) plan a build-stage `build-tool-depends` (e.g. alex) was therefore compiled by the native build ghc (linux ELF) but named via `exeTargetName (hostPlatform lbi)` with the Host platform's `.exe` suffix — so a linux `alex` was installed as `alex.exe`, and the consumer's configure (searching bare `alex`) failed with `[Cabal-7620] program 'alex' could not be found`. Patch the fork to take `plat`/`compil` from the package's own stage toolchain, matching the program db. No-op for single-stage/native plans (Build == Host). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The stable-haskell GHC bootstrapped via haskell.nix's own ghc9103, which is built from source (~32 derivations on darwin). nixpkgs' ghc9103 is the same GHC version and is fully cached in cache.nixos.org (0 builds), so route the boot compiler through the `compilerSelection` hook to nixpkgs (`p.haskell.compiler`) — the same nixpkgs-boot pattern already used for v2-cabal-install and the iserv-proxy tools. Applied to stage1, the alex/happy/genprimopcode/deriveConstants boot tools, `buildGHC`, the cross `deriveConstants`/`libffi-wasm` tools, and the dev shells. Verified by dry-run: the boot ghc is now fetched, not built. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
… fixes Task #15 (stable-haskell -target two-stage cross). Gated behind `isTwoStagePlan` / `hostPlatform.isWindows` so native, hadrian and single-stage-cross slice hashes stay byte-identical (the one exception is `targetSupportsShared`, which also fixes the dynamic-way decision for static/musl cross targets). - comp-v2-builder: `buildToolSourceFrags` contributes a build-tool's SOURCE frag (tarball + .cabal) to a consuming slice instead of composing the tool's cyclic built slice; drop build-tool-only ids from store composition for two-stage plans (avoids host-exe→host-lib eval recursion); expose `v2SourceFrag`; `--via-asm` hsc2hs for Windows (Win32 pointer-cast #enums). - build-cabal-slice: accept `buildToolSourceFrags`, stage them into the slicing repo (`extraFragStaging`; empty → hash-neutral otherwise). - call-cabal-project-to-nix: two-stage host `--with-compiler` reports the target platform so os()/arch() conditionals resolve (else Windows plans drop Win32 and every slice re-solve fails "unknown package: Win32"). - cabal-project: inject Win32 hackage source as a local packages: entry on Windows targets; key the boot-lib `shared:` off the target platform (`targetSupportsShared`); Win32 +os-string flag. Complements the already-committed alex.exe fix (224e400): the source injection here is what the InLibrary exe-naming patch operates on. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
…platform
The boot switch (previous commit) routes the boot compiler through nixpkgs'
`p.haskell.compiler`, which — unlike a haskell.nix compiler — has no
`evalWith.${evalSystem}` variant. lib/dummy-ghc.nix reads `ghc.raw-src` to
source boot-package .cabal files during plan-to-nix, and that source
derivation is system-specific. So when Hydra evaluates a foreign-system job
on the eval host (the x86_64-linux ghc914-sh jobs are evaluated on
aarch64-darwin), the boot ghc's raw-src was the *build* platform's
(x86_64-linux) — dragging an x86_64-linux realisation into the eval-host
plan-to-nix.
Override `raw-src` in `bootCompilerSelection` to the eval-platform compiler's
copy (GHC source is platform-independent, so the plan content is identical);
the actual build compiler's outPath is untouched by `//`, so it stays
build-platform. No-op when evalSystem == build system: verified the native
aarch64-darwin ghc914-sh compiler drvPath is unchanged
(0c8qbkqyjdw502b5jpgqaj3i97bsk1hg).
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The stable-haskell two-stage cross plans (`withBuildCompiler`) were selecting
a target-hosted compiler here via `compilerSelection pkgs` (`pkgs` =
`final.buildPackages`, target = the cross target). Forcing that compiler
evaluates its host=target splice, which the ghcjs backend rejects outright
("attempt to build ghcjs with ghcjs", overlays/bootstrap.nix) and which never
makes sense for any cross target.
These plans don't need a target-hosted compiler at all: the plan-to-nix dummy
already reports the cross target from `mkDummyGhcPkg`'s `pkgs`, and target-OS
`os()`/`arch()` conditionals (e.g. Win32) are handled by the boot-package
injection in modules/cabal-project.nix. Drop the special case so `ghc'` uses
the build-hosted compiler uniformly.
Verified: aarch64-darwin ghc914-sh ghcjs.hello now evaluates (was throwing);
musl64.hello and native ghc914-sh drvPaths byte-identical.
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
`overlays/wasm.nix` rebuilds the LLVM `monorepoSrc` (a plain source-tree copy: `buildPhase = "true"`, `cp -r . $out`) so it can apply the haskell-wasm patch. It used `final.stdenv` — the wasm *cross* stdenv, whose `cc` is the wasm clang. But that clang is built FROM this very `monorepoSrc`, so the reference closed a bootstrap cycle (clang → monorepoSrc → final.stdenv → clang → …) that evaluated to an infinite recursion. On aarch64-darwin this made every wasi32 job (e.g. ghc914-sh.wasi32.hello) fail to evaluate with "infinite recursion encountered" reached through stdenv-darwin's defaultNativeBuildInputs. A source copy needs no target compiler: build it with `final.buildPackages.stdenv` (native, build platform), which breaks the cycle. Verified: pkgsCross.wasi32.buildPackages.llvmPackages.clang and the full wasi32.hello job now evaluate (previously infinite recursion). Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
…pkgs
GHC's wasm backend still requires the haskell-wasm wasi-libc fork (ghc-wasm-meta
ships a forked wasi-sdk and states "Upstream wasi-sdk won't work yet"; the
patches are not upstreamed). The pinned rev (951e93b) and LLVM (21.1.8, via the
haskell-wasm llvm-project patch) already match exactly what the current wasi-sdk
fork uses — nothing was stale.
The nixpkgs bump broke the *build* of that fork in three ways, all now handled
in the `wasilibc` override:
1. nixpkgs re-packaged its own wasilibc around CMake (wasi-sdk-32).
`overrideAttrs` inherited cmake/ninja and their setup hooks, so
cmakeConfigurePhase ran against the fork tree (Makefile only, no
CMakeLists.txt) and aborted. Drop cmake/ninja, disable their hooks, and
drive the fork's own `make default libc_so` with explicit
configure/build/install phases — reusing the base derivation only for its
stdenvNoLibc + cross-toolchain libc-bootstrap wiring.
2. We build through nixpkgs' cc-wrapper (ghc-wasm-meta uses the raw wasi-sdk
clang), which injects `-rtlib=compiler-rt`; clang-21 flags it as unused on
`-c` steps and the Makefile compiles with `-Werror`. Demote it via
NIX_CFLAGS_COMPILE (appended after the Makefile's flags). Also drop
CHECK_SYMBOLS=yes: that self-test diffs clang's predefined macros against a
set captured with the raw wasi-sdk clang, and clang-21 adds benign extras
(_LIBCPP_HARDENING_MODE) that fail the diff without affecting the libc.
3. nixpkgs' wasilibc lost its `share` output and enabled structuredAttrs;
install via the $out/$dev shell vars and skip the sysroot/share metadata
(build-verification output, not needed by libc consumers).
Verified on aarch64-darwin: pkgsCross.wasi32.wasilibc and the full wasm clang
wrapper toolchain now build.
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
The Hydra eval of this branch failed outright: the `nix-tools` aggregate
(flake.nix) lists `x86_64-darwin.nix-tools.static.zipped.*` constituents, and
resolving them forced evaluation of x86_64-darwin under the bumped
nixpkgs-unstable, which now throws "Nixpkgs 26.11 has dropped support for
x86_64-darwin". Hydra reported the constituents as "does not exist" and failed
the whole jobset (no eval produced).
26.05 is the last nixpkgs release supporting x86_64-darwin, so rather than drop
the platform, pin it there:
* Add a `nixpkgs-2605` input (26.05-darwin); refresh `nixpkgs-2511`.
* ci.nix: the "Nixpkgs version" matrix uses R2605 (replacing R2511) plus
`unstable`, but `unstable` is excluded on x86_64-darwin (importing 26.11
there throws). So x86_64-darwin builds only the R2605 dimension.
* flake.nix: `legacyPackages`/`legacyPackagesUnstable` fall back to the 26.05
pin on x86_64-darwin (unstable can't be imported there); every downstream
output (packages/checks/devShells/requiredJobs) inherits the working pkgs.
* The x86_64-darwin static nix-tools constituents stay: the ./nix-tools
subflake resolves pkgs via its own pinned haskellNix, whose unstable nixpkgs
predates the drop and still supports x86_64-darwin. The original failure was
only the `callFlake` forcing `self.legacyPackages.x86_64-darwin` (26.11),
which the fallback above fixes.
Verified on aarch64-darwin: the `nix-tools` aggregate, x86_64-darwin static
nix-tools, x86_64-darwin R2605 roots/requiredJobs, and the aarch64-darwin /
x86_64-linux R2605+unstable dimensions all evaluate.
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
nixpkgs 26.05/26.11 renamed several attrs that lib/pkgconf-nixpkgs-map.nix
resolves, so every eval printed:
evaluation warning: 'xxHash' has been renamed to 'xxhash'
evaluation warning: 'QuadProgpp' has been renamed to 'quadprogpp'
evaluation warning: 'librest_1_0' has been renamed to/replaced by 'librest'
`lookupAttrsIn` force-evaluates each mapped attr, which triggers the deprecated
alias' warning. Point the entries at the new names, but keep working on older
nixpkgs where only the old names exist:
* xxHash->xxhash, QuadProgpp->quadprogpp: use the existing `prefer new old`
helper. The new name appears exactly when the old one becomes an alias
(both at 26.05), so `prefer` picks the new name on 26.05+ and the still-real
old name on <=25.11 — warning-free on every pin.
* librest_1_0->librest: `prefer` is wrong here because `librest` exists on
every nixpkgs as a *distinct* older package (0.8.x, the 0.7 API) that does
not provide `rest-1.0.pc` — that is why the map used `librest_1_0`. The 1.0
provider was renamed to `librest` and `librest_1_0` became a deprecated
alias only in 26.11, so gate on the nixpkgs release (`lib.version`): real
`librest_1_0` through 26.05, `librest` from 26.11 on.
Verified across pins (24.11/25.05/25.11/26.05/unstable): the three warnings are
gone on the CI pins (R2605 + unstable) and each older pin resolves to the
correct, still-present attribute.
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
head.hackage is a rolling repository, so the fixed-output derivation that runs `cabal v2-update` against it goes stale whenever head.hackage updates. Refresh the --sha256 to the current content hash. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
* lib/call-cabal-project-to-nix.nix: for a stable-haskell CROSS plan the `-w` dummy must report the TARGET platform (else every arch(javascript)/ os(ghcjs) conditional resolves for the build host) and present an EMPTY global package db so the plan builds rts/base/... from source, as the fork's boot-package injection expects. Gated on `isStableHaskell`: a mainline compiler's cross plan needs the opposite (non-empty dump, boot libs pre-existing) and keeps the pre-existing behaviour. * lib/dummy-ghc.nix: match the fork's real `--info` output so tests.dummy-ghc-info holds: the fork brands `Edition: Stable Haskell`, and every ghc914-sh (native and each -target wrapper) is the SAME stage-2 binary, so `Stage` is 2 and `RTS ways` is the baked-in `v thr debug thr_debug` across all targets. Mainline compilers keep their per-target values. * test/dummy-ghc-info: ignore `ld supports verbatim namespace` — a per-target linker capability cabal does not consult for elaboration. * modules/install-plan/redirect.nix: make-install-plan emits targets that are not buildable in the current environment as bare status STRINGS in `available`; extend the filter to TargetDisabledBySolver and TargetNotLocal (e.g. a package marked `buildable: False` for the JS target) — previously `defaultTargetId` forced `.id` on the reason string and eval failed with "expected a set but found a string".
Two-stage (stable-haskell -target cross) plans build their build-tools from SOURCE inside the consuming slice. Complete that machinery and gate the JS-backend specifics; all changes are byte-identical for slices with no build-tool source frags and for non-ghcjs targets. * comp-v2-builder.nix (buildToolSourceFrags): a tool exe built build-stage in-slice needs its own LIBRARY deps in the slicing repo too (happy 2.x exe:happy build-depends happy-lib). Walk each tool exe's dep closure on the BUILD-twin-preferring plan index and source-stage the configured units; exclude the GHC boot-lib set by name — the fork reinstalls boot libs from source in both stages, so neither `type` nor nonReinstallablePkgs distinguishes them, and their build-stage src is a null-hash boot fetchurl that coerces on staging. nonReinstallablePkgs threaded in via hspkg-builder.nix. * comp-v2-builder.nix (ghcjs): the ghcjs platform has no stdenv.cc, so merely realizing a native C lib / framework / pkgconfig dep from the target pkg-set throws "no C compiler provided for this platform". A JS slice links no host C libraries — empty `libs`/`frameworks`/`pkgconfig` at the source and skip the transitiveDepLibs / pkgcfglessSysLibs walks. * comp-v2-builder.nix (pkgTarball): keep the direct-`src` shortcut only when src is provably a hackage tarball (name ends .tar.gz — their slice hashes don't move); otherwise normalize through a v2-tar runCommand that packs a DIRECTORY source (a source-repository-package SUBDIR such as Cabal-syntax in the stable-haskell Cabal SRP) into a deterministic tarball. Previously the raw directory was symlinked as `.tar.gz` and slice staging failed `tar: ...: Cannot read: Is a directory`. * build-cabal-slice.nix: build-stage tool sub-builds inside a ghcjs-target slice need a NATIVE toolchain the slice stdenv doesn't provide — add the build-platform cc wrapper + libiconv (darwin `-liconv` via LIBRARY_PATH), and make every composed platform db's package.cache writable so the build-stage `ghc-pkg recache` doesn't fail flock on a store symlink. ghcjsArWrapper now execs emar with `--format=gnu` (llvm-ar's darwin default counts 8-byte alignment padding IN the ar size field, which wasm consumers read as trailing garbage — "section too large"). * ghc-shim.nix: the `ar command` settings sed expected `", "` but the settings file has no space after the comma — the swap never matched and the ar wrapper was never used. Match optional whitespace and assert the swap took. (This shim only engages for prefixed-ghc targets; the ghc914-sh JS wrapper's targetPrefix is "" and gets the equivalent fix in its own settings assembly.)
* targetSupportsShared: the JS backend is static-only — emscripten's wasm-ld rejects the ELF soname flag GHC emits when linking a boot lib's shared object (`wasm-ld: error: unknown argument: -h`). Exclude ghcjs, mirroring the Windows/musl exclusions, so the injected `package *` config sets `shared: False`. * Template Haskell: GHC's JS external interpreter resolves the `ghci` package BY NAME in the home unit state at splice time (GHC/Runtime/Interpreter/JS.hs lookupPackageName) and links GHCi.Server.defaultServer from its unit closure. `ghci` is never a build-depends, so no slice's dep db contains it, and the two-stage wrapper's target global db is empty — the lookup failed for every TH-using package. Populating the compiler's GLOBAL db with a separately-built ghci closure does NOT work: its unit-ids can never match the project's own boot-lib builds, so the interpreter session loads ghc-internal's jsbits twice (server closure + splice deps) and node dies "Identifier 'h$base_o_rdonly' has already been declared". Instead compose THE PROJECT'S OWN ghci slice into every host package's starting store db via additional-prebuilt-depends (ghcjs only), exactly like the rts WAY sub-libs above: the closure propagates with the slice and unit-ids agree with the splice code by construction. ghci's own dependency closure is excluded by name to avoid derivation cycles (none of those packages run TH — they all build before ghci exists). * ghc-internal: its configure guards the JS sizeof/offset checks with `test "$host" = "javascript-ghcjs"`, but cabal's runConfigureScript passes the full triple `--host=javascript-unknown-ghcjs`, so the block silently skipped and HsBaseConfig.h shipped `#undef SIZEOF_STRUCT_STAT` etc. — the jsbits then die at TH time with "ReferenceError: SIZEOF_STRUCT_STAT is not defined". A prePatch makes the guard accept both spellings. (Proper fix belongs upstream in stable-haskell/ghc libraries/ghc-internal/configure.ac.)
Assorted fixes to the javascript-unknown-ghcjs `-target` wrapper, all gated isGhcjsTarget; other cross targets (wasm etc.) are byte-identical. * settings `JavaScript CPP command`: ghc-toolchain-bin resolves every tool to an absolute path EXCEPT this one (left as bare `emcc`, which is not on slice PATHs) — sed it to the same absolute emcc every other command uses. * settings `ar command`: emar is llvm-ar, whose DEFAULT archive format on a macOS build host is darwin — member data 8-aligned with '\n' padding COUNTED IN the ar size field. Mach-O readers tolerate the inline pad; wasm ones parse to the entry's declared end and fail "section too large" (llvm-nm, and wasm-ld at the TH-interpreter and final links, on e.g. time's HsTime.o). Point the setting at a small emar wrapper forcing `--format=gnu` (exact member sizes); cabal takes its `ar` from this settings entry too, so the one swap covers both GHC-internal and cabal archive creation. * bin/nm: nixpkgs emscripten ships no `bin/emnm`; the raw emnm.py self-locates via abspath(__file__) for `from tools import shared`, so a symlink breaks it. Generate a wrapper from the nixpkgs emar wrapper (same env setup) that execs emnm.py by its real path, and assert the substitution matched. * ghc-interp.js: TH splices run under node against this interpreter script, resolved at `$topdir/ghc-interp.js` — install it into the target registration. * enableShared = false for ghcjs (static-only backend). * mkRawDrv now takes an optional `populatedDb` (register a prebuilt target boot-lib closure into the global db, with the confs' pkgroots rewritten absolute). The ghcjs wrapper itself stays the plain empty-db crossCompilerBoot: TH-on-JS is handled at the project level by the boot-package injection (see modules/cabal-project.nix) — an earlier "bootstrap tier" that baked a separately-built ghci closure into the global db could never share unit-ids with the project's own boot libs, and the JS interpreter then loaded ghc-internal's jsbits twice. * rts-configure tool discovery: targetBintoolsUnprefixed exposes unprefixed nm/objdump for ghc-toolchain; boot-lib harvest helpers (findComponent over versioned hsPkgs keys) support the populatedDb path.
The new evaluator runs in impure mode, so `builtins.currentSystem` resolves to the actual builder's system rather than the intended fixed eval platform. Hardcode `evalSystem = "aarch64-darwin"` to keep it deterministic. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
`test/setup-deps/default.nix` takes `evalSystem` as a required argument, but
its call site in `test/default.nix` imported it directly (bypassing `callTest`,
which threads `evalSystem`) and never passed it. Under the impure Hydra
evaluator — where `evalSystem` no longer defaults to `builtins.currentSystem` —
forcing this job throws:
error: function 'anonymous lambda' called without required argument 'evalSystem'
| { pkgs, compiler-nix-name, evalPackages, evalSystem }:
Pass `evalSystem` at the call site. Verified by forcing every test lambda for
ghc984 and ghc914-sh under a non-native `evalSystem` (67 tests each, 0 failures).
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
…oject-local store, LLD)
* build-cabal-store-sync (shell-for-v2.nix) + compose-store.nix: the
stable-haskell cabal fork's v2 slices use the staged
host/<platform>/{package.conf.d,units,lib} store layout, not the old
ghc-<ver>/package.db. The sync script scanned only ghc-*/ and so
installed nothing; teach it (and the compose recache) the host layout,
add the native build->host stage link, and target the project-local
<builddir>/store (the fork keeps its store there, not ~/.cabal/store) —
taking BUILDDIR as an arg (default dist-newstyle).
* shell-for-v2.nix: let a `tools` value be a prebuilt derivation (used
as-is), so a project can drop in the v2 fork cabal
(haskell-nix.v2-cabal-install) rather than mainline cabal.
* stable-haskell.nix: on darwin, pass ghc-toolchain-bin
--cc-link-opt=-fuse-ld=<ld64.lld> so ghc914-sh's settings link with LLD.
The dev-shell/v2 cctools ld (ld64-956.6 + clang/llvm libLTO) SIGTRAPs on
any Cocoa link; find_ld / ghc-toolchain refuse to search for lld on
darwin (GHC #21712) and the build sets --disable-ld-override, so we set
the link flag directly (the ghc-toolchain analogue of useLdLld's
CONF_GCC_LINKER_OPTS_STAGE2).
Every conflict was one shape: master added the `testCabalProjectLocal` / `testInputMap` test arguments (the locally built head.hackage, #2560) where this branch had added `evalSystem`. Resolved as the union of both. - test/default.nix: fold `evalSystem` into master's `intersectAttrs` mechanism for optional test arguments, rather than keeping this branch's separate `wantsEvalSystem` check -- one mechanism instead of two. - build.nix: master takes `evalPackages` as an argument, which this branch replaced with `evalSystem`. The automatic merge combined this branch's signature with master's new head-hackage wiring, leaving `evalPackages` unbound; bind it from `evalSystem` via the memoised per-eval-system nixpkgs. - flake.nix: keep both sides' new inputs (hyper-linux, and head-hackage / hackage-overlay-repo-tool); take master's removal of the `hydraJobs.nix-tools` aggregate, which supersedes this branch's tweak to it. Verified: every test function applies for ghc984, ghc914-sh and a forced non-native evalSystem=x86_64-linux (68 tests, 0 failures, at ifdLevel 2 so no IFD), ci.nix composes for all seven unstable compilers, and shell-for's shell derivation evaluates through the newly built head.hackage. Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| # compilers) need this to suppress the "WARNING: ghc.cachedDeps not found" | ||
| # trace and to give make-config-files.nix correct envDeps/exactDeps data. | ||
| haskell-nix = prev.haskell-nix // { | ||
| compiler = __mapAttrs (name: ghc: |
Contributor
There was a problem hiding this comment.
[diff] reported by reviewdog 🐶
Suggested change
| compiler = __mapAttrs (name: ghc: | |
| compiler = __mapAttrs (_name: ghc: |
| # attribute is lazy, so on build hosts that never cross-compile to Linux (or use | ||
| # qemu) it is never forced. | ||
| { sources }: | ||
| (final: prev: |
Contributor
There was a problem hiding this comment.
[diff] reported by reviewdog 🐶
Suggested change
| (final: prev: | |
| (_final: prev: |
ghc914-sh's ghcjs cross builds succeeded but every JS executable it
produced aborted at startup:
Aborted('HEAP8' was not exported. add it to EXPORTED_RUNTIME_METHODS)
at h$initEmscriptenHeap -> Module.onRuntimeInitialized -> run
Newer emscripten no longer exports HEAP* by default, and GHC's JS RTS
shim reads `Module.HEAP8`, so `rts/js/mem.js` has to name it in its EMCC
`EXPORTED_RUNTIME_METHODS` pragma. haskell.nix already carries that fix
as overlays/patches/ghc/ghc-9.12-ghcjs-rts-mem-heap8.patch -- but that
list lives in overlays/bootstrap.nix and only reaches the *hadrian*-built
compilers. ghc914-sh is built by overlays/stable-haskell.nix straight
from this pin, so it never saw it. Only build-only ghcjs jobs are green;
anything that runs the output fails, which is why it went unnoticed.
Rather than teach stable-haskell.nix a second patch list, the fixes now
live on the branch: stable-ghc-9.14-hn is stable-ghc-9.14 plus
- JS: export HEAP8 symbol (#26290) \ cherry-picked from the fork's
- JS: export HEAPU8 (#26290) / master via feat/js-export-heap8
- compiler: trim Apple ranlib's padding from wasm archive members
The third has no upstream equivalent on either stable-ghc-9.14 or master
(master's 894a04f3a8 is already in 9.14 and fixes archive member size
*writing*, not this padding *reader*), so it is committed on the branch.
The third ghcjs patch in that range, ghc-9.13-ghcjs-rts-types.patch, is
NOT needed: stable-ghc-9.14 already lists rts/Types.h in rts.cabal.
Verified through the pin that both fixes are present in the fetched
source. The functional check -- rebuilding the compiler and running
ghcjs hello under node -- is a full compiler rebuild and is still in
flight; it is also the first compile-test of the Ar.hs change.
Note the flake-compat narHash gotcha: `nix flake update` records a hash
that flake-compat's fetch disagrees with, so the lock carries the
FOD-reported hash instead.
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
…xes)
Ports the rest of the overlays/bootstrap.nix GHC patches that apply at
9.14 onto the fork. That list is only consulted for the hadrian-built
compilers, and stable-haskell builds ONE multi-target compiler from this
pin rather than one GHC per target, so a target-guarded patch list cannot
serve it -- the fixes have to be in the source. Merged as three feature
branches:
feat/aarch64-elf-linker-pool contiguous linker pool so aarch64 ADRP
relocations stay in range under qemu
(GHC #24432); inert on aarch64-darwin,
which is Mach-O not ELF
feat/windows-rts-linker-fixes MinGW RTS symbols time/unix need, and
keep an import symbol's own SymType on
PEi386 lookup
feat/android-bionic-support ELF{32,64}_ST_VISIBILITY fallback for
bionic, bare `android` OS in
GHC_CONVERT_OS, COMPAT_R_ARM_PREL31
relocation, skip loading libc.a
Two deviations from the carried patches, both deliberate:
- The PEi386 fix assigned `*type` unguarded where that function
otherwise always checks `type`, so a caller passing NULL (lookupSymbol
does) would segfault. Guarded in the ported commit. The hadrian
compilers still apply the unguarded version.
- ghc-9.12-static-linker-script-support.patch is NOT ported. It is 70
lines across GHC/Linker/{Static,Unit,Loader}.hs and SysTools/Ar.hs; one
hunk fails outright and the others only match with fuzz 2-3 at offsets
up to -219 lines, i.e. at the wrong sites. Those modules have moved
too far since 9.12 for a mechanical port, and a wrong port here is a
compiler compile error. It needs a hands-on port -- or a check that it
is still needed at all. Tracked in the audit.
Verified through the pin that all eight ported fixes are present in the
fetched source. narHash is again the FOD-reported one, not what `nix
flake update` records (flake-compat disagrees with it).
Claude-Session: https://claude.ai/code/session_015iMn9nFmfb5DPqqW5a12co
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds
haskell-nix.compiler.sghc914, a GHC 9.14 compiler built from thestable-haskell/ghc
stable-ghc-9.14branch using haskell.nix's own
cabalProjectmachinery — no hadrian. Itreplicates the two-stage bootstrap from that branch's
Makefile/cabal.project.*files:ghc9103)the final cross compiler is assembled from native stage1 binaries +
target settings (via
ghc-toolchain) + the target package DBEvery cabal component is its own derivation, so:
ghc9141on aarch64-darwin (~38% smaller).Verified
helloviahackage-package)hello.wasmruns under wasmtimehydraJobs...hello.drvPath); builds not yet runsghc914is added toci.nix(unstable,runTests = true).How it works
lazy-inputs/):sghc914(the GHC tree, no submodules —upstream dropped them) plus
sghc914-cabal/sghc914-hpc-bin/sghc914-hsc2hsfor thesource-repository-packages the project filesreference, wired into the stage projects via
inputMap(no rev fetching ateval time, works with the moving
stable-haskell/masterCabal branch).overlays/stable-haskell.nix(new, ~1900 lines) holds everything:configuredSrc(autoreconf/configure + project-file merging), the threecabalProject'instances, compiler assembly, and target-specific fixes(wasm libffi, ghcjs via emscripten, Windows dual-universe handling — see
below).
as direct hackage tarball URLs are no longer in the GHC tree, so the
synthesised
ghc-pkg dumpwould miss them. AdumpSrcview (configuredtree + those packages'
.cabalfiles) is exposed via the final compilers'raw-srcpassthru so user-project plans see the true installed set.New general-purpose features
replace-hackage-tarball-urlsproject option (default off): rewriteshttps://hackage.haskell.org/package/NAME-VER/NAME-VER.tar.gzentries in aproject's
packages:stanza to local store paths fetched via hackage.nix(
haskell-nix.hackageTarball), so plan-to-nix needs no network and thepinned versions need not exist at the project's
index-state.haskell-nix.hackageTarball { name; version; }: fetch + unpack an exacthackage release using the sha256 hackage.nix already knows.
haskell-nix.sghc914-shells):from-boot(boot GHC as
GHC0,make stage1from a clean tree) andfrom-stage1(haskell.nix's cached stage1 as
STAGE1_PATH, rebuild only stage2 withcabal). Consumed by a
flake.nixin the GHC checkout.Mainline fixes included (each is a standalone commit, cherry-pickable into separate PRs on request)
modules/install-plan/redirect.nix: treatTargetDisabledByUserlikeTargetNotBuildable— previously a package whose only plan target wasdisabled (e.g.
unixon Windows) crashed evaluation with"expected a set but found a string".
lib/system-nixpkgs-map.nix: GHC 9.14 rts / ghc-internal extra-libraries —c/rt/dlon Linux;wsock32/dbghelp/psapi/ntdll/ucrt/mingw32on Windows.
overlays/haskell.nix: guardhadrianProject/iserv-proxy-exesexistence in
roots(sghc914 has neither).Windows notes (the interesting part)
On Windows targets cabal resolves the
Win32 → hsc2hs → process → Win32build-tool cycle by solving a second, older universe inside the
Win32:hsc2hs:exescope (old Win32 predatesbuild-tool-depends). This iscorrect solver behaviour — pinning
any.hsc2hsmakes the plan unsatisfiable —so the overlay tolerates it: boot libraries are registered into the compiler
DB by versioned key, and the exe-scope duplicates stay private to the tool
build. Cross-TH (
templateHaskell/iserv-proxy-exes) required registeringthe compiler attr in cross-host package sets and cutting a recursion in
crossBootProject(it builds the libraries the compiler is assembled from,so it can never use the TH wrapper itself).
Known issues / follow-ups
9.14while the overlay pins9.14.0(
ghcVersionFull) — reconcile if version-keyed paths misbehave.-mavx2per-file-CMM-flags issue; Windows / ghcjs / android builds not yet run
(plans verified only); wine/qemu iserv TH untested.
How to test
Update — renamed
ghc914-sh, darwin→linux cross, v2 slice builder (2026-07)Since the original description this PR has grown; latest
masteris merged in.Renamed
sghc914→ghc914-shThe compiler attr,
ci.nix, the dev shells, and thelazy-inputs/pins now useghc914-shthroughout. Compilers built withcabalProjectsetpassthru.isStableHaskell = true.darwin → linux cross-compilation via hyper-linux (
hl)aarch64-darwinbuild hosts can now cross-compile toaarch64-unknown-linux-musl(fully static) and run the resulting Linux ELFs —including the Template Haskell iserv interpreter — directly on Apple Silicon via
Hypervisor.framework. qemu user-mode doesn't exist on macOS, so
hlfills the"emulator" slot in the TH/test path:
overlays/linux-cross.nixgains apipeCommandargument (defaults toqemu-<suffix>; when passed, qemu is never forced) — the wrapper/wrapGhclogic is otherwise emulator-agnostic.
overlays/armv6l-linux.nixselects${pkgsBuildBuild.hyper-linux}/bin/hlaspipeCommandon darwin build hosts.lib/default.nixisNativeMuslnow requiresbuildPlatform.isLinux(darwin'slinuxArchis alsoarm64, so darwin→aarch64-musl was misclassified asnative musl).
Verified end-to-end (compile + run + Template Haskell under
hl) for bothghc914-shand the hadrianghc9141—helloandtest/th-dlls-minimalrun asstatic aarch64 Linux ELFs and drive the full iserv GHCi message loop. The hadrian
path needed one extra fix (
compiler/ghc/default.nix): a targetlibffi.dylibso ld64 links stage0 correctly when cross-compiling from darwin.
Caveats / follow-ups (this side stays a PoC for now):
hlis not yet pinned —armv6l-linux.nixexpectshyper-linuxto beprovided as a
pkgsBuildBuildoverlay attr (it throws otherwise). Pinning itas
lazy-inputs/hyper-linuxis the remaining integration step, plus addingaarch64-multiplatform-musltoci.nixforaarch64-darwin.nix buildTH is currently blocked by an upstreamhlSIGBUS whenspawned under the nix-daemon on this host; TH was verified by running the
build phases via
nix develop <drv>as the login user. Needs an upstreamhlfix.
x86_64-unknown-linux-muslcompiles fine from Apple Silicon but doesn'trun under
hlon M1 (x86_64 guests need Rosetta's 48-bit IPA VM — anM2+ requirement).
v2 slice builder aligned with
ghc914-shThe incremental per-component ("slice") builder now reproduces plan-nix's
UnitIds exactly for stable-haskell compilers, so real projects build againstghc914-shon the v2 builder. This needed:9.14(it was forking9.14.0vs9.14) —this resolves the version-reconciliation follow-up noted above;
ghc --infotruthful aboutGHC Dynamicand RTSways for statically-linked stable-haskell GHCs;
--ipidboot-library ids + a-Bon the assembled stage2ghcwrapper (sounlitresolves);style: "local"/"inplace"packages (the generated haskell-gi
gi-*family), whose-inplaceids theslice reproduces as global hashes by construction.
Exercised end-to-end by building the leksah
dev shell against this branch:
ghc 9.14+cabal-install 3.16.1.0+haskell-language-server 2.14.0.0all resolve and the shell enters.Closes #347: because
ghc914-shis assembled from per-packagecabalProjectoutputs, the RTS and each boot library are their own store paths, so downstream closures can reference just the subset they need (the 2.4 GiB vs 3.9 GiB reduction above) — which is what "put the RTS in a separate output" was after.