Reconcile the cross/dual-compiler stack onto stable-haskell/master (rpath/tool-guess/stage-qualified)

[angerman]

Reconcile the cross / dual-compiler stack onto stable-haskell/master

The fixes the stable-haskell GHC build depends on are scattered across pre- and
post-split feature branches and none are on master. As a result
stable-haskell/ghc#188 has
to pin the pre-split feature/wasm-cross-ghcup-stack, which forces GHC to
revert compiler/Setup.hs + libraries/ghc-boot/Setup.hs to the
pre-VerbosityHandles API and to drop the hooks-exe sub-dir at stage0. This
tracks getting everything onto master so GHC can pin master and drop those
adaptations.

Two generations, split by the Verbosity commit edb808a0b8b

branch	Verbosity split	host-only / stage-qual	#361 tool-guess	#368 rpath	self-$ORIGIN
master (01d053ac)	✅	(stage-qual code present)	❌	❌	❌
feature/cross-compile, wip/andrea/stage-qualified-options	✅	✅ (#378)	❌	❌	❌
feature/wasm-cross-ghcup-stack (GHC's current pin)	❌	✅	✅	✅	✅

The pin line and the post-split line share no recent ancestor; the integrated,
working stack is on the old line, while the modern API is on master.

Reconciliation — converge on master via the stage-qualified line

• cabal-install: stage-qualified package configuration (package build:*) #378 wip/andrea/stage-qualified-options → master (open, WIP) — the
  stage-qualified / host-only package * handling. The integration trunk;
  everything else stacks on or lands beside it.
• GHC/Build/Link: relativize absolute rpaths + self-$ORIGIN (stacked on #378) #385 rpath relativize + self-$ORIGIN — stacked on cabal-install: stage-qualified package configuration (package build:*) #378
  (draft). Supersedes GHC/Build/Link: relativize absolute rpaths when --enable-relocatable #368 (feat/rpath-relativize-absolute, stranded
  against the pre-split feature/rebase-CI) and the master-targeting GHC/Build/Link: relativize absolute rpaths + self-$ORIGIN (forward-port onto master) #383
  (closed). De-risked: cherry-picks onto the post-split line with no
  conflicts; only needs a build to confirm it typechecks against the
  post-split LocalBuildInfo/InstallDirs API. Rides onto master when
  cabal-install: stage-qualified package configuration (package build:*) #378 lands.
• Try target-prefixed ghc-pkg first in guessToolFromGhcPath #361 fix/target-prefix-aware-tool-guess → master (open) — cross
  ghc-pkg guess (independent; GHC.hs).
• Remove dead GHCJS import + fix stale binDirectoryFor call #359 fix/ghcjs-and-bindirectory-removal → master (open).

Downstream once the above are on master

In stable-haskell/ghc:

• Re-pin cabal.project.stage{0,1,2.common,3} from
  feature/wasm-cross-ghcup-stack to master.
• Revert the Setup.hs Verbosity revert (restore the
  #if MIN_VERSION_Cabal(3,17,0) form — master has VerbosityHandles again).
• Restore the hooks-exe sub-dir at stage0.

The self-$ORIGIN rpath fix (#385) also removes glibc's reliance on the runtime
LD_LIBRARY_PATH crutch, so it's a genuine improvement, not only a musl
workaround (root cause of backpack/cabal/T14304 failing on Alpine/musl).

[angerman]

Local stage0→stage2 validation of the post-split pin (finding)

Built stable-haskell/ghc against feat/wasm-cross-ghcup-stack-next
(ee3b55e6 = wip/andrea/stage-qualified-options #378 + #361 + #368 + #385's
$ORIGIN), with both pre-split hacks dropped (Setup.hs revert reverted,
hooks-exe restored). Results on aarch64-darwin:

• stage0 ✅ — the post-split Cabal 3.17.0.1 compiles + links with
  bootstrap ghc-9.8.4 → working cabal-install. The GHC/Build/Link: relativize absolute rpaths + self-$ORIGIN (stacked on #378) #385 rpath forward-port
  compiles against the post-split LocalBuildInfo/InstallDirs API (the open
  question on GHC/Build/Link: relativize absolute rpaths + self-$ORIGIN (stacked on #378) #385 — answered: it's fine).
• stage1 ✅ — compiler/Setup.hs + ghc-boot/Setup.hs compile in their
  upstream #if MIN_VERSION_Cabal(3,17,0) VerbosityHandles form against the
  post-split lib (log shows VerbosityFlags {vLevel = Normal …} in use).
  ⇒ dropping the Setup.hs revert is correct on the post-split line.
• stage2 (DYNAMIC=1) ❌ — build:os-string is built shared and fails:
  ld: library not found for -lHSbytestring-0.12.2.0-…-ghc9.8.4.

Root cause (verified, not inferred)

Only the pin changed — cabal.project.stage2.dynamic is byte-identical to
the one that is green on aa57a49. So the regression is purely the cabal
side: aa57a49 makes a bare package * { shared: True } implicitly
host-only (build-stage deps excluded); #378's stage-qualified-options model
does not — the unqualified package * now applies to all stages, so
shared: True leaks onto build-stage packages and they fail to link a shared
bootstrap lib that doesn't exist.

Implication for reconciliation

Re-pinning GHC to the post-split line is not a drop-in. In addition to #378
landing, GHC's cabal.project.stage2.{dynamic,common,…} must be rewritten to
#378's stage-qualified syntax to scope shared/executable-dynamic/
library-for-ghci to the host/target stage only. Either that, or #378 should
preserve the implicit host-only default for an unqualified package *.

Until #378 stabilizes (its syntax is still moving) and that adaptation is
written, stable-haskell/ghc stays on the pre-split aa57a49 pin — it is the
only branch that builds GHC green across platforms. (stage0/stage1 being green
means the only remaining blocker is this host-only/stage-qualified mismatch.)

[angerman]

Resolved the stage2 blocker — modern pin now builds GHC green locally.

Applying #378's own fixture pattern to GHC's cabal.project.stage2.dynamic fixes it:

package host:*
  shared: True
  executable-dynamic: True
package build:*
  shared: False
  executable-dynamic: False

With that, stage0 + stage1 + stage2 (DYNAMIC=1) build green on aarch64-darwin, and stage2 ghc is dynamically linked (@rpath/libHSghc-9.14-ghc9.14.dylib). So the post-split pin is viable for GHC; the only project-file adaptation needed is stage-qualifying the shared/executable-dynamic settings (an unqualified package * leaks onto the build stage). The stage3 (cross) if os(wasi) { package * shared: True } block will likely need the same host:*/target:* treatment — not yet validated.

Still gated on #378 finalizing (its stage-qualified syntax is the dependency).

[angerman]

stage3 wasm-cross on the modern pin: a second, deeper blocker (beyond host-only).

With the stage2 host:*/build:* fix in place, the wasm cross build now gets through stage2 + stage3 rts + compiles all 298 ghc-internal modules + installs the library — then fails in the post-install shared-lib finalize step:

[GHC-74335] -dynamic is ignored when linking binaries on WASM
Exception when reading interface file .../ghc-internal/.../AllocationLimitHandler.dyn_hi
mismatched interface file profile tag (wanted "", got "dyn")

Root: if os(wasi) { package * shared: True } builds dyn-only interfaces (.dyn_hi), but since -dynamic is ignored when linking on WASM the finalize step runs GHC vanilla and wants a vanilla .hi. The pre-split pin shipped wasm with shared: True fine, so this is a post-split-cabal -dynamic-too / cross-interface behavior change (same family as the JS .dyn_hi gap) — not a host-only package * leak, and not a one-line project-file fix. Needs cabal/GHC-side investigation of how the shared wasm lib's interfaces are produced (vanilla + dyn, i.e. -dynamic-too).

So on the modern pin: native (stage0-2) is green; wasm-cross has this additional blocker.

[angerman]

Update — wasm-cross on the modern pin: validated the hard part, but it requires the full #181 GHC-source+build stack (not just the cabal pin).

Drove stage3-wasm on the modern-pin branch through several layers (each a fix #181 feat/wasm-cross-ghcup carries that stable-ghc-9.14/#188 lacks):

1. cabal.project.stage3 if os(wasi){shared} → Rebase failed on 2025-12-09 #181's if arch(wasm32){shared+executable-dynamic+rts+dynamic} (else ghc-internal link [GHC-74335]).
2. rts/RtsStartup.c: guard the promoteBootLibrariesToGlobal() call with !defined(wasm32_HOST_ARCH) (matches its wasm-excluded definition).
3. Ported Rebase failed on 2025-12-09 #181's two compiler commits 4d84acee93f (per-target targetIsDynamic settings) + 7396909b730 (GHC/Linker/Dynamic.hs wasm dynamic-link) — this fixed [GHC-74335]: ghc-internal now builds clean for wasm on the modern pin. ✅
4. Next failure: host:ghc-9.14 configure → genprimopcode: parse error. genprimopcode/primops.txt.pp/Setup.hs are identical to Rebase failed on 2025-12-09 #181, so the cause is the settings generation (Makefile + ghc-toolchain + mk/multi-target-* — Rebase failed on 2025-12-09 #181 commits not yet ported): the ported compiler reads per-target settings, but without the build-system that writes them the wasm settings yield bad cppOpts.

Conclusion: the native modern-pin is fully validated (stage0-2 green). wasm-cross is not a cabal-pin problem — it needs #181's complete wasm stack (compiler ✅ + rts ✅ + build-system ❌). The efficient path is to validate wasm-modern on the #181 base (re-pin #181 to the post-split cabal) rather than reconstruct #181's stack on #188. This is the #181 ↔ stable-ghc-9.14 ↔ #378 integration.

[angerman]

Post-split cabal-install ignores active-repositories: :none for build-tool-depends — a reproducibility regression (found via wasm, but general).

On the #181 base + modern pin, stage3's genprimopcode build pulls happy from Hackage (Downloading hackage.../happy-2.1.5.tar.gz) even though cabal.project.stage3 sets active-repositories: :none and provides happy as an explicit URL. The Hackage happy's generated Parser.hs is internally inconsistent (happyDoParse signature+use at 1144/1164 but the binding is happyParse at 1222 → GHC-44432), which the pinned-URL happy would not produce. Pinning happy ==2.1.5, happy-lib ==2.1.5 did not help — the source still came from Hackage.

This is not wasm-specific: a post-split cabal-install that resolves build-tools from Hackage despite :none breaks the hermetic-build guarantee the GHC build relies on. Worth a dedicated cabal issue.

Net on wasm-cross-on-modern-pin (after ~12 build iterations across #188 and #181): it's a deep, multi-layer integration, not a quick validation. Layers found & fixed: (1) cabal.project.stage3 arch(wasm32) form, (2) RtsStartup.c call guard, (3) compiler per-target-settings + Linker/Dynamic — fixes GHC-74335, ghc-internal builds for wasm ✅, (4) process 1.6.26.1→1.6.29.0, (5) happy/:none (open). The native modern-pin is green on both bases; the hard wasm part (GHC-74335) is validated-fixed. The rest is dep/build-tool reconciliation with the post-split cabal.

[angerman]

Consolidation — two GHC-build-config bugs surfaced while validating the post-split re-pin (both happy-2.1.7-triggered, both fixed/identified):

1. stage1 hermeticity gap — cabal.project.stage1 lacked active-repositories: :none + a happy pin (stage2/stage3 have them), so build-tools floated to Hackage; happy-2.1.7's release then broke native CI across platforms (genprimopcode GHC-44432). Fixed: stable-haskell/ghc PR Rebase failed on 2025-12-17 #190 (standalone, → stable-ghc-9.14) + applied on Rebase failed on 2025-12-16 #189. Latent base bug, not modern-pin-specific.
2. devx PATH shadows pinned build-tools (cross only) — the devx env puts happy-2.1.7/alex-3.5.4.0 on PATH; the cross stage3 build-stage uses those instead of the pinned 2.1.5/3.5.2.0 (the native stages correctly use the plan's). Fix = stage3 Makefile must put the pinned tools ahead of devx's PATH (or --with-happy/--with-alex). Validated by PATH-prepend.

Net: native modern-pin validated (stage0-2 green with the hermeticity fix). Full wasm-cross on the modern pin remains a multi-layer integration — after the above, genprimopcode now builds+runs but hits a wasm primops-preprocessing parse error (primops.txt 139:31), still to crack. The cabal-side reconciliation here (#385 stacked on #378, etc.) is unaffected by these GHC-build-config bugs.