Skip to content

fix(agent-image): ship the uv binary in the runtime stage (#68) - #88

Merged
stubbi merged 1 commit into
mainfrom
fix/agent-image-ship-uv-binary
Jun 18, 2026
Merged

fix(agent-image): ship the uv binary in the runtime stage (#68)#88
stubbi merged 1 commit into
mainfrom
fix/agent-image-ship-uv-binary

Conversation

@stubbi

@stubbi stubbi commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Second half of the #68 fix

#85 added pyproject.toml/uv.lock at /opt/venv-template/, which let init-uv get past its cp step — and immediately exposed a second latent defect in the same init container.

The conformance suite (un-skipped in #87, now that #85 republished the image) caught it:

pod/conformance-minimal-0   0/1   Init:Error   4 restarts
  init-uv  Terminated  Exit Code: 127
  logs init-uv: /bin/sh: 1: uv: not found

init-uv runs uv sync --frozen at pod start to materialise the env onto the per-instance PVC, but the runtime stage copies the resolved venv (/opt/venv) and never the uv binary. So uv sync → exit 127. Before #85, init-uv died at the earlier cp, masking this.

Fix

  • COPY --from=uv /uv /uvx /usr/local/bin/ in the runtime stage (mirrors the builder).
  • Deterministic UV_* defaults so the pod-start sync never downloads a Python or hardlinks across the image-layer → PVC boundary: UV_PYTHON_DOWNLOADS=never, UV_PYTHON=/usr/local/bin/python, UV_LINK_MODE=copy.

Verification (local)

Built the image and ran the exact init-uv command:

$ docker run --rm --entrypoint /bin/sh hermes-agent:uvtest -c \
    'set -eu; cd /home/hermes/.hermes; cp /opt/venv-template/{pyproject.toml,uv.lock} .; uv --version; uv sync --frozen'
uv version: uv 0.11.7
...
=== EXIT 0 ; .venv populated ===

Exit 127 → exit 0; .venv is created on the (PVC-equivalent) workdir.

After merge

Republish ghcr.io/paperclipinc/hermes-agent:v2026.5.29.2, then #87's conformance Idempotency job should go green and the un-skip lands.

Refs #68

🤖 Generated with Claude Code

Second half of the #68 init-uv contract fix. #85 added the lockfiles at
/opt/venv-template/, which let `init-uv` get past its `cp` step and reach
`uv sync --frozen` — which then failed with exit 127 ("uv: not found"),
because the runtime stage copies the resolved venv (/opt/venv) but never the
uv binary. (Before #85, init-uv died at the earlier `cp`, masking this.)

Copy uv/uvx into the runtime image (mirroring the builder stage) and set
deterministic UV_* defaults so the pod-start `uv sync` never tries to download
a Python or hardlink across the image-layer -> PVC boundary:
  - UV_PYTHON_DOWNLOADS=never
  - UV_PYTHON=/usr/local/bin/python
  - UV_LINK_MODE=copy

Verified locally: building the image and running the exact init-uv command
(cp + uv sync --frozen) now completes and populates the per-instance .venv.

Refs #68

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@stubbi
stubbi enabled auto-merge (squash) June 18, 2026 17:25
@stubbi
stubbi merged commit 13fb1e5 into main Jun 18, 2026
14 checks passed
@stubbi
stubbi deleted the fix/agent-image-ship-uv-binary branch June 18, 2026 17:28
stubbi added a commit that referenced this pull request Jun 18, 2026
Un-skipping (earlier in this branch) proved the #68 init-uv contract is fixed
(#85 lockfiles, #88 uv binary, image republished): init-uv now succeeds and the
pod reaches its main hermes container. But the diagnostics dump showed the
instance still can't reach Ready in CI, for reasons beyond #68:

  1. hermes-agent exits immediately without an LLM provider+credentials
     ("No LLM provider configured"); CI fixtures have none -> CrashLoopBackOff.
  2. The long-lived agent form is `hermes gateway run` (a chat-platform daemon
     binding no :8443 server), not the one-shot `hermes-agent run` the entrypoint
     invokes, so the operator's TCPSocket :8443 readiness probe can't pass as-is.
  3. The published image is missing modules (websockets, hermes_cli.dashboard_auth).

Reaching Ready in CI needs design decisions (inject LLM creds, rework
entrypoint/readiness/port, fix image deps), not just the #68 image fix. So keep
the corpus skipped, but refresh the reason (a new idempotencyReadyBlockedSkip
const) to reflect that #68 is resolved and point at the real blockers (#64). The
waitForInstanceReady diagnostics dump is retained so the next attempt is
debuggable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stubbi added a commit that referenced this pull request Jun 18, 2026
…p diagnostics on timeout (#87)

Follow-up to #85 and #88. **Net effect: the Ready-gated idempotency
corpus stays skipped, but for accurate reasons, plus a diagnostics
dump.** (This branch briefly un-skipped the corpus to find out what was
actually blocking Ready — that investigation is summarized below.)

## What the investigation found

#68 (the `init-uv` contract) is **fixed**: #85 ships the lockfiles at
`/opt/venv-template/`, #88 ships the `uv` binary, and `v2026.5.29.2` was
republished + verified. `init-uv` now succeeds and the pod reaches its
main `hermes` container.

But un-skipping showed the instance still can't become Ready in CI —
three deeper, separate blockers (surfaced by the new diagnostics dump):

1. **No LLM provider** — the published hermes-agent exits immediately
without a provider + credentials (`Failed to initialize agent: No LLM
provider configured`). CI fixtures carry none → container exits 0 →
CrashLoopBackOff.
2. **Wrong serve command / port model** — the long-lived agent is
`hermes gateway run` (a chat-platform daemon that binds **no** `:8443`
server), not the one-shot `hermes-agent run` the image entrypoint
invokes. The operator's `TCPSocket :8443` readiness probe can't pass
as-is.
3. **Missing image deps** — `websockets`, `hermes_cli.dashboard_auth`.

Reaching Ready in CI needs design-level decisions (inject LLM creds,
rework the entrypoint/readiness/port model, fix image deps) — out of
scope for the #68 image fix.

## This PR

- Keeps the corpus skipped, but replaces the stale `#68` skip reason
with `idempotencyReadyBlockedSkip` describing the real blockers (and
noting #68 is resolved). Tracked under #64.
- Adds `dumpInstanceDiagnostics` to `waitForInstanceReady` (instance
YAML, pods/sts, describe, events, init-container logs) so the next Ready
attempt is debuggable from the CI log instead of a bare timeout.

Conformance stays green (skips). Un-skip once an instance can actually
reach Ready in CI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stubbi pushed a commit that referenced this pull request Jun 19, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.1.18](v0.1.17...v0.1.18)
(2026-06-19)


### Features

* **runtime:** run agents on the upstream s6 image — instance reaches
Ready (gateway API + /health)
([#90](#90))
([12dccb4](12dccb4))


### Bug Fixes

* **agent-image:** ship pyproject.toml + uv.lock at /opt/venv-template/
([#68](#68))
([#85](#85))
([39ef52c](39ef52c))
* **agent-image:** ship the uv binary in the runtime stage
([#68](#68))
([#88](#88))
([13fb1e5](13fb1e5))
* **backup:** bound the on-delete finalizer so a failing snapshot can't
make an instance undeletable
([#94](#94))
([40d6099](40d6099))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: paperclip-release-bot[bot] <288053502+paperclip-release-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant