fix(spawn): map full id range for nested/in-pod builds so image-owned files copy up (#432)#433
Merged
Merged
Conversation
… files copy up (#432) A nested `pelagos build` RUN step (uid 0 without CAP_SYS_ADMIN — a k8s pod, the #426 path) could not create files inside image directories owned by a non-root uid/gid. cargo build was the visible casualty: error: failed to acquire package cache lock failed to open: /usr/local/cargo/.package-cache Value too large for defined data type (os error 75) # EOVERFLOW Root cause (isolated end-to-end on an ext4 node): the nested user namespace mapped only `0→0`. Container image layers own files as arbitrary ids — rust's `/usr/local/cargo` is gid 983. overlayfs copy-up must preserve that ownership onto the new upper file, but an id that is unmapped in the namespace cannot be represented, so *any* create beneath such a directory fails with EOVERFLOW. It is not cargo-specific (plain `touch` fails identically), not the flock, not the file size, and not `userxattr` — the real pelagos mount does not even use userxattr. cargo is simply the first thing a real build does that writes under a non-root-owned image directory (its package-cache lock). Fix: in the nested uid-0 case, identity-map the whole id space instead of just `0→0`. We still hold CAP_SETUID/CAP_SETGID there (only CAP_SYS_ADMIN was dropped), so the parent writes a multi-count map — the same mechanism already used for root-created user namespaces. This keeps in-pod builds fully rootless on native overlay (no privileged pod, no fuse-overlayfs perf hit). True rootless (uid != 0) is unchanged: it keeps subuid-range mapping, falling back to the single-id map when no subuid delegation exists. Both the `spawn` and `spawn_interactive` map-setup sites are updated; a new `has_effective_cap` helper generalises the existing CAP_SYS_ADMIN probe. Validated end-to-end: the real nested `rust:1-bookworm` + `cargo build` RUN step that failed with EOVERFLOW now completes and produces an image, rootless on ext4. Reproduction and root-cause/fix isolation scripts land in scripts/. Adds `test_nested_build_overlay_copyup_unmapped_gid` (drops CAP_SYS_ADMIN, builds an overlay whose lower has a gid-983 directory, creates a file inside it) and its entry in docs/INTEGRATION_TESTS.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #432. A nested
pelagos buildRUN step (uid 0 withoutCAP_SYS_ADMIN— a k8s pod, the #426 path) could not create files inside image directories owned by a non-root uid/gid.cargo buildwas the visible casualty:Root cause (isolated end-to-end on an ext4 node)
The nested user namespace mapped only
0→0. Container image layers own files as arbitrary ids — rust's/usr/local/cargois gid 983. overlayfs copy-up must preserve that ownership onto the new upper file, but an id unmapped in the namespace cannot be represented, so any create beneath such a directory fails withEOVERFLOW.Everything else was a red herring, including the issue's own framing:
openat, before anyflockuserxattr— the real mount doesn't even use itxino/ cross-device — inodes ~10M, same deviceO_LARGEFILE— plaintouch(glibc) fails identicallyFix
In the nested uid-0 case, identity-map the whole id space instead of
0→0. We still holdCAP_SETUID/CAP_SETGIDthere (onlyCAP_SYS_ADMINwas dropped), so the parent writes a multi-count map — the same mechanism already used for root-created user namespaces. Keeps in-pod builds fully rootless on native overlay (no privileged pod, no fuse-overlayfs perf hit). True rootless (uid ≠ 0) is unchanged (keeps subuid-range mapping). Bothspawnandspawn_interactivemap-setup sites updated.Validation
rust:1-bookworm+cargo buildRUN step that failed with EOVERFLOW now completes and produces an image, rootless on ext4 (ipc4).0→0map →CREATE-FAIL (EOVERFLOW); identity range map →CREATE-OK.test_nested_build_overlay_copyup_unmapped_gid(+ doc);cargo test --lib392 passed; canonicalcargo clippy -- -D warningsclean.scripts/repro-432-*.sh.🤖 Generated with Claude Code