comp-builder: add per-component buildJobs option (#1479) - #2546
Open
hamishmack wants to merge 1 commit into
Open
comp-builder: add per-component buildJobs option (#1479)#2546hamishmack wants to merge 1 commit into
hamishmack wants to merge 1 commit into
Conversation
Parallel building was unconditionally on, with the v1 builder baking `-j$(($NIX_BUILD_CORES > 4 ? 4 : $NIX_BUILD_CORES))` into `Setup build` and hardcoding `enableParallelBuilding = true`. Memory-hungry packages had no first-class way to reduce or disable this. Add a nullable `buildJobs` component option (modules/component-options.nix): * null (default) -> unchanged capped `-j` behaviour * 1 -> sequential build (`-j1`, enableParallelBuilding off) * N -> pinned `-jN` Wire it through builder/comp-builder.nix (buildJobsFlag + enableParallelBuilding). The default (option unset) produces a byte-identical build phase and `enableParallelBuilding = true`, so existing derivations are unchanged. Scoped to the v1 builder: v2 slices explicitly cannot reproduce per-component differentiation faithfully (see comp-v2-builder.nix), and a slice may cover multiple components, so a per-component job count does not map onto v2. Adds test/build-jobs asserting (at eval level, no compile) that the component buildPhase and enableParallelBuilding reflect the option for the default, buildJobs=1 and buildJobs=3 cases.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a per-component buildJobs option to the v1 component builder so individual components can pin or disable Cabal parallelism (and align enableParallelBuilding), addressing the resource-usage concerns raised in #1479.
Changes:
- Introduces
packages.<pkg>.components.<kind>.<name>.buildJobs(nullable positive int) inmodules/component-options.nix. - Wires the option into the v1 builder (
builder/comp-builder.nix) via a computedbuildJobsFlagandenableParallelBuilding = buildJobs != 1. - Adds an eval-level test (
test/build-jobs) asserting the resultingbuildPhasestring andenableParallelBuildingfor default /1/3.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| test/default.nix | Registers the new build-jobs test in the test suite. |
| test/build-jobs/default.nix | Adds eval-only assertions for buildJobs effects on buildPhase and enableParallelBuilding. |
| modules/component-options.nix | Defines the new per-component buildJobs module option and its documentation/type. |
| builder/comp-builder.nix | Implements the v1 builder behavior: constructs -j flag from buildJobs and toggles enableParallelBuilding. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+20
to
+28
| description = '' | ||
| Number of jobs to pass to `Setup build` as `-jN` for this component. | ||
| `null` (the default) keeps the standard behaviour of `-j` capped at | ||
| `min(NIX_BUILD_CORES, 4)`. Set to `1` to build sequentially (useful | ||
| for memory-hungry packages that OOM under parallel builds), or to any | ||
| other positive integer to pin the parallelism. Only affects the v1 | ||
| (`builderVersion = 1`) builder. | ||
| ''; | ||
| type = lib.types.nullOr lib.types.ints.positive; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a first-class, per-component
buildJobsoption so memory-hungry packages can reduce or disable parallel building — the long-standing request in #1479.The v1 builder previously hardcoded
enableParallelBuilding = true(builder/comp-builder.nix) and baked-j$(($NIX_BUILD_CORES > 4 ? 4 : $NIX_BUILD_CORES))into theSetup buildinvocation, with no way to override short of an undocumented trailing-j1insetupBuildFlags.Change
New nullable option in
modules/component-options.nix:null(default) → unchanged:-jcapped atmin(NIX_BUILD_CORES, 4),enableParallelBuilding = true1→-j1andenableParallelBuilding = falseN→ pinned-jN,enableParallelBuilding = trueWired through
builder/comp-builder.nixvia abuildJobsFlagbinding andenableParallelBuilding = buildJobs != 1.Default behaviour is byte-identical when the option is unset: the emitted flag string is exactly the original literal and
enableParallelBuildingstaystrue(null != 1), so no existing derivation changes hash.Scope: v1 only
Deliberately scoped to the v1 builder. The v2 builder (
comp-v2-builder.nix) documents that it cannot faithfully reproduce per-component differentiation, and a v2 slice may build several components at once, so a per-component job count doesn't map cleanly onto v2. The option's description says so.Tests
Adds
test/build-jobs(registered intest/default.nix). It builds thecabal-simpleproject three ways (default,buildJobs = 1,buildJobs = 3) and asserts — at eval level, without compiling — on the resulting exe component'sbuildPhasestring andenableParallelBuilding:-jexpression andenableParallelBuilding = truebuildJobs = 1emits-j1, drops the capped default, and setsenableParallelBuilding = falsebuildJobs = 3emits-j3and keepsenableParallelBuilding = trueVerification done
nix-instantiate --parsecleanly.nix-instantiate test/default.nix --argstr compiler-nix-name ghc9124 -A build-jobs.runsucceeds, i.e. the module option is accepted and all 7 eval assertions pass (they guard therunderivation viaassert, so instantiation would abort if any failed).true;1→-j1+false;8→-j8+true.Closes #1479.