Skip to content

fix: share an explicit DOCKER_CONFIG between docker login and compose so private pulls work under ProtectHome - #80

Open
eav93 wants to merge 1 commit into
Finsys:mainfrom
eav93:fix/registry-login-protecthome
Open

fix: share an explicit DOCKER_CONFIG between docker login and compose so private pulls work under ProtectHome#80
eav93 wants to merge 1 commit into
Finsys:mainfrom
eav93:fix/registry-login-protecthome

Conversation

@eav93

@eav93 eav93 commented Jun 26, 2026

Copy link
Copy Markdown

Problem

Deploying a stack that pulls a private image fails with:

Error response from daemon: Head "https://ghcr.io/v2/<org>/<image>/manifests/latest": unauthorized

even when the registry credentials are correct.

ComposeClient.Execute runs docker login (via loginToRegistries) and then docker compose up/pull. docker login persists credentials to $HOME/.docker/config.json, but the bundled systemd unit (scripts/hawser.service) hardens the agent with ProtectHome=true, which makes $HOME (/root) unwritable inside the service's mount namespace. As a result:

  • docker login cannot persist the credentials (and the error was only logged at debug level, so it was invisible);
  • the subsequent compose command — which is given a clean environment (commit 3cbc419) — finds no credentials and pulls anonymously → unauthorized.

This affects every private registry on a default install, not just GHCR. It reproduces reliably with the shipped unit; docker login/docker compose succeed only when run by hand outside the sandbox (where /root/.docker is visible).

Fix

Give docker login and the compose command a shared, writable DOCKER_CONFIG:

  • create an ephemeral DOCKER_CONFIG directory under stacksDir (already writable per the unit's ReadWritePaths) for up/pull operations that carry registry credentials, and RemoveAll it when the operation completes;
  • pass that DOCKER_CONFIG to both docker login and the compose command through a small dockerEnv helper. The helper also gives docker login the same minimal, clean environment the compose command already uses, so host vars such as DOCKER_CONTEXT/DOCKER_TLS_VERIFY cannot redirect it;
  • raise a failed docker login from debug to warn, since it is the usual root cause of a later unauthorized pull.

No systemd unit change is required.

Notes / tradeoffs

  • The credential file lives only for the duration of the operation, in a 0700 dir owned by root, and is removed via defer. On a hard crash/kill a .docker-* dir could linger; left as an accepted tradeoff to keep this change small (a sweep of stale dirs could be a follow-up).
  • stacksDir is MkdirAll'd before use so custom installs without a pre-created stacks dir still work.
  • The fix depends only on stacksDir, not on $HOME, so it also works for non-systemd deployments (e.g. the Docker image), including containers with a read-only root filesystem, where $HOME/.docker would be unwritable too. When stacksDir is unset the behaviour is unchanged.

Testing

  • go build ./..., go vet ./..., go test ./... pass.
  • Reproduced on a host running the agent with ProtectHome=true: a clean env with an unwritable $HOME makes docker compose pull of a private image fail with the exact unauthorized error; running docker login with an explicit shared DOCKER_CONFIG and then compose with the same DOCKER_CONFIG pulls successfully.

docker login persisted credentials to $HOME/.docker, but the bundled systemd
unit runs the agent with ProtectHome=true, which makes $HOME (/root) unwritable
inside the service's mount namespace. The credentials were therefore never
stored, and the subsequent `docker compose` pull — run with a clean
environment — fell back to anonymous access and failed for private registries
with "unauthorized". The login failure was only logged at debug level, so it
was invisible.

Create an ephemeral DOCKER_CONFIG directory under stacksDir (already writable
via the unit's ReadWritePaths) for up/pull operations and share it between
`docker login` and the compose command through a small dockerEnv helper. The
helper also gives `docker login` the same minimal, clean environment the
compose command already uses, so host vars like DOCKER_CONTEXT/DOCKER_TLS_VERIFY
cannot redirect it. Failed logins are now surfaced at warn level. No systemd
unit change is required.
@eav93
eav93 force-pushed the fix/registry-login-protecthome branch from 7f28b17 to b373ab7 Compare June 26, 2026 23:47
@dschoepel

Copy link
Copy Markdown

Hit this exact bug independently on a self-hosted Dockhand/Hawser setup (agent v0.2.46, Ubuntu, systemd unit with ProtectHome=true): private ghcr.io pulls fail with unauthorized even with correct registry credentials configured via DOCKER_CONFIG=/etc/docker-credentials. Currently working around it by pre-pulling private images manually on the host before deploying via Dockhand — would love to drop that workaround. This PR's description matches the root cause exactly (docker login silently failing to persist under ProtectHome, then a clean-env compose subprocess pulling anonymously). Any chance of getting this reviewed/merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants