fix(spawn): keep setgroups=allow in nested user namespaces so apt/su work (#434)#435
Merged
Merged
Conversation
…work (#434) After #432 removed the privileged-pod workaround, a rootless nested `pelagos build` (uid 0 without CAP_SYS_ADMIN — a k8s pod) failed on any RUN step that calls setgroups(2). apt was the visible casualty: E: setgroups 0 failed - setgroups (1: Operation not permitted) E: Method http has died unexpectedly! pelagos: error: RUN command failed with exit code 100 Cause: the parent id-map writer set /proc/<pid>/setgroups=deny before writing the child's gid_map. That deny is only required for the unprivileged own-id mapping. In the nested build the ROOT parent writes the map and holds CAP_SETGID, so it can write the gid_map while setgroups stays "allow" — which is what Docker/Podman/runc do for range-mapped build user namespaces. Denying it needlessly makes setgroups(2) inside the container return EPERM. Latent until now because the old privileged pod skipped the user namespace entirely. Fix: in both parent id-map writer sites (spawn + spawn_interactive), only write setgroups=deny when we do NOT hold CAP_SETGID; otherwise leave it allow. Validated on an ext4 node: `FROM debian RUN apt-get install ...` under `capsh --drop=cap_sys_admin` fails before, succeeds after. Unblocks the full in-cluster gruesome build (cargo #432 + apt) with no privileged pod. Adds `test_nested_build_setgroups_allowed` (+ docs) — backend-independent, asserts the nested userns leaves /proc/self/setgroups=allow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #434. After #432 removed the privileged-pod workaround, a rootless nested
pelagos build(uid 0 withoutCAP_SYS_ADMIN) failed on any RUN step that callssetgroups(2)— notablyapt-get:Cause
The parent id-map writer set
/proc/<pid>/setgroups=denybefore writing the child's gid_map. That deny is only needed for the unprivileged own-id mapping. In the nested build the root parent writes the map and holdsCAP_SETGID, so it can write the gid_map whilesetgroupsstaysallow(what Docker/Podman/runc do for range-mapped build user namespaces). Latent until now — the old privileged pod skipped the userns entirely.Fix
Both parent id-map writer sites (
spawn+spawn_interactive): only writesetgroups=denywhen we do not holdCAP_SETGID; otherwise leave itallow.Validation
FROM debian/RUN apt-get install ca-certificates) undercapsh --drop=cap_sys_adminon an ext4 node: fails before, succeeds after (Fetched … / Successfully built).test_nested_build_setgroups_allowed(backend-independent);cargo test --lib392 passed;cargo clippy -- -D warningsclean.pelagos build: cargo (flock) fails with EOVERFLOW on the userxattr overlay #432 + apt).🤖 Generated with Claude Code