Skip to content

fix(agent): nsenter mode finds host binary via /proc/1/root - #144

Merged
sarg3nt merged 1 commit into
mainfrom
fix/nsenter-host-binary-path
May 18, 2026
Merged

fix(agent): nsenter mode finds host binary via /proc/1/root#144
sarg3nt merged 1 commit into
mainfrom
fix/nsenter-host-binary-path

Conversation

@sarg3nt

@sarg3nt sarg3nt commented May 18, 2026

Copy link
Copy Markdown
Owner

Summary

Distroless agent images don't ship util-linux, so the existing nsenter mode bailed at exec time with

pty: start failed: exec: "nsenter": executable file not found in $PATH

on mjolnir under the privileged + pid:host deploy.

With pid: host the kernel exposes the host's root filesystem at /proc/1/root (PID 1's magic root symlink resolved by the host's mount namespace). The agent can call the host's nsenter binary via that path without bundling util-linux into the distroless image — and without changing any deployment plumbing.

Changes

  • resolveHostNsenter() stat()s a small candidate list and returns the first match (/proc/1/root/usr/bin/nsenter, /proc/1/root/bin/nsenter, etc., with /usr/bin/nsenter as an in-container fallback for non-distroless agent flavors).
  • SpawnNsenter uses the resolved absolute path as argv[0] instead of "nsenter".
  • nsenterUsable() now requires the resolver to succeed, so the capabilities envelope honestly reports host_console: false when no reachable nsenter exists. Dashboard would rather hide the action than offer a shell that immediately fails.

Test plan

  • go test ./... green on the agent module
  • Deploy on mjolnir: rebuild + docker compose -p gearbox-agent ... up -d, then click Mjolnir shell icon — expect level=INFO msg="console: nsenter host-exec selected" followed by a session opening and landing on the host (hostname returns the TrueNAS hostname, not the agent container).

🤖 Generated with Claude Code

Distroless agent containers ship no util-linux, so the previous
SpawnNsenter argv of {"nsenter", "--target", "1", ...} bailed at exec
with "executable file not found in \$PATH" — observed live on mjolnir
running ghcr.io/sarg3nt/gearbox/gearbox-agent:latest:

  level=ERROR msg="console: spawner failed"
   error="pty: start failed: exec: \"nsenter\": executable file not found in \$PATH"

With `pid: host` the kernel exposes the host's root filesystem at
/proc/1/root (PID 1's magic root symlink resolved by the host's mount
namespace). The agent can call the host's nsenter binary via that path
without bundling util-linux into the distroless image.

resolveHostNsenter stat()s a small candidate list:
  /proc/1/root/usr/bin/nsenter   (Debian/Ubuntu/modern RHEL)
  /proc/1/root/bin/nsenter       (older trees)
  /proc/1/root/usr/sbin/nsenter
  /proc/1/root/sbin/nsenter
  /usr/bin/nsenter               (in-container fallback for non-distroless
                                  agent flavors)

nsenterUsable now requires the resolution to succeed so the capabilities
envelope honestly reports `host_console: false` when no nsenter is
reachable — the dashboard would rather hide the action than offer a
shell that immediately fails.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 18, 2026 22:44
@github-actions github-actions Bot added the agent label May 18, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes nsenter host-exec mode in the distroless agent image by locating the host's nsenter binary via /proc/1/root (exposed under pid: host) instead of relying on the container's $PATH.

Changes:

  • Added resolveHostNsenter() that stats a candidate list under /proc/1/root plus /usr/bin/nsenter fallback.
  • SpawnNsenter now uses the resolved absolute path as argv[0] and errors out early if no binary is reachable.
  • nsenterUsable() requires resolveHostNsenter() to succeed so host_console capability is honestly reported.

@sarg3nt
sarg3nt merged commit cb6b52b into main May 18, 2026
23 checks passed
sarg3nt added a commit that referenced this pull request May 19, 2026
…troless (#145)

* fix(agent): bundle busybox nsenter — host's binary can't load in distroless

PR #144 made stat() of /proc/1/root/usr/bin/nsenter succeed but execve
still failed with ENOENT, observed live on mjolnir:

  pty: start failed: fork/exec /proc/1/root/usr/bin/nsenter:
    no such file or directory

Root cause: the host's nsenter is dynamically linked. Its PT_INTERP
(e.g. /lib64/ld-linux-x86-64.so.2) is resolved by the kernel against
the CALLER's mount namespace, not against PID 1's. The distroless
container has no /lib64 and no glibc, so the ELF loader can't find
the interpreter and execve returns ENOENT (the misleading error
message refers to the missing interpreter, not the binary itself).

Fix: bundle a statically-linked nsenter inside the agent image.
busybox-musl ships a single static binary that dispatches applets
by argv[0]; copying /bin/busybox to /usr/bin/nsenter inside the
agent image gives us a working nsenter that exec's cleanly in any
mount namespace. ~1MB image-size cost; agents not using host-exec
pay it once but never run it.

Resolver: reorder candidate list so container-local paths
(/usr/bin/nsenter etc.) come first; /proc/1/root paths remain as
last-ditch fallback for non-distroless agent flavors that happen
to share enough libc layout with the host.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* address Copilot review findings on PR #145

- Pin busybox by image-index digest
  (sha256:19b646668802469d968a05342a601e78da4322a414a7c09b1c9ee25165042138)
  so the build is reproducible across upstream tag rebuilds. buildx
  multi-arch resolver picks the right platform manifest from the
  index automatically. Verified busybox 1.37.0's nsenter applet
  supports -t/--target, -m/--mount, -u/--uts, -i/--ipc, -n/--net,
  -p/--pid — the exact flags SpawnNsenter emits.
- Restore /proc/1/root/usr/sbin/nsenter and /proc/1/root/sbin/nsenter
  to the candidate list so the host-fallback search mirrors the
  container-local list (which already includes /usr/sbin and /sbin).
- "binary not found" error now reports the actual list searched
  (built from the candidates slice) instead of the stale
  "/proc/1/root and /usr/bin" string.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@sarg3nt
sarg3nt deleted the fix/nsenter-host-binary-path branch May 28, 2026 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants