Skip to content

Commit 125b6ae

Browse files
stubbiclaude
andcommitted
test(conformance): un-skip the Ready-gated idempotency corpus (runtime fixed)
The runtime blockers (#68#89) are fixed in #90: the operator runs the upstream s6 hermes-agent image with `gateway run` + the OpenAI API server and probes HTTPGet /health. The agent image has been republished FROM upstream and the fixtures are bumped to ghcr.io/paperclipinc/hermes-agent:v0.16.0 (the new runtime, verified: /init entrypoint, HERMES_HOME=/opt/data, multi-arch). Un-skip the 9 #68-gated idempotency corpus entries; the conformance suite now exercises real Ready + reconcile-idempotency against the published image. The ollama-webterminal-tailscale entry stays skipped — it still needs a live ephemeral Tailscale auth key (unrelated; #64). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 12dccb4 commit 125b6ae

11 files changed

Lines changed: 27 additions & 40 deletions

test/conformance/idempotency_test.go

Lines changed: 17 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -14,44 +14,31 @@ import (
1414
// more times. After each requeue we assert the resourceFingerprint is unchanged
1515
// (generation + resourceVersion must not move). This catches lesson #437
1616
// regressions: a reconciler that always re-writes owned objects will fail here.
17-
// The Ready-gated corpus stays skipped only until the agent image is republished.
18-
//
19-
// The runtime blockers that previously made Ready unreachable are FIXED in the
20-
// operator code (PR #90, issue #89): the operator now runs the upstream s6
21-
// hermes-agent image with `gateway run` + the OpenAI API server, probes
22-
// HTTPGet /health, and injects a placeholder LLM provider so the gateway comes
23-
// up without live calls (validated end-to-end on kind — an instance reaches
24-
// Ready=True). The old failure modes are gone: no one-shot `hermes-agent run`,
25-
// no TCPSocket :8443 with nothing listening, no LLM-credential requirement just
26-
// to start, no missing modules (the upstream image ships everything incl. a
27-
// browser).
28-
//
29-
// The one thing the conformance suite still needs is the *published* image to BE
30-
// that upstream-based runtime: these fixtures pin
31-
// `ghcr.io/paperclipinc/hermes-agent:v2026.5.29.2`, which only ships the new
32-
// runtime after the FROM-upstream Dockerfile (PR #90) is merged and that tag is
33-
// republished. Un-skip in the follow-up once the image is republished.
34-
// (waitForInstanceReady dumps pod diagnostics on timeout if anything regresses.)
35-
const idempotencyReadyBlockedSkip = "runtime fixed in #90 (upstream s6 image + gateway run + /health, validated on kind); un-skip once ghcr.io/paperclipinc/hermes-agent:v2026.5.29.2 is republished FROM upstream"
17+
// The Ready-gated corpus now runs. The runtime blockers (#68 → #89) are fixed in
18+
// #90: the operator runs the upstream s6 hermes-agent image with `gateway run` +
19+
// the OpenAI API server, probes HTTPGet /health, and injects a placeholder LLM
20+
// provider so the gateway comes up without live calls. The fixtures pin
21+
// `ghcr.io/paperclipinc/hermes-agent:v0.16.0`, which is the upstream-based
22+
// runtime. Validated end-to-end on kind (an instance reaches Ready=True).
23+
// waitForInstanceReady dumps pod diagnostics on timeout if anything regresses.
3624

3725
var idempotencyCorpus = []struct {
3826
label string
3927
fixture string
4028
// skip, when non-empty, skips this corpus entry with the given reason.
4129
// Used for fixtures that cannot reach Ready in CI for reasons unrelated to
42-
// operator idempotency (e.g. they require live external credentials, or are
43-
// blocked by an out-of-scope operator bug).
30+
// operator idempotency (e.g. they require live external credentials).
4431
skip string
4532
}{
46-
{label: "minimal", fixture: "minimal.yaml", skip: idempotencyReadyBlockedSkip},
47-
{label: "maximal", fixture: "maximal.yaml", skip: idempotencyReadyBlockedSkip},
48-
{label: "gateways-all", fixture: "gateways-all.yaml", skip: idempotencyReadyBlockedSkip},
49-
{label: "selfconfig-enabled", fixture: "selfconfig-enabled.yaml", skip: idempotencyReadyBlockedSkip},
50-
{label: "profilestore-enabled", fixture: "profilestore-enabled.yaml", skip: idempotencyReadyBlockedSkip},
51-
{label: "autoupdate-enabled", fixture: "autoupdate-enabled.yaml", skip: idempotencyReadyBlockedSkip},
52-
{label: "backup-enabled", fixture: "backup-enabled.yaml", skip: idempotencyReadyBlockedSkip},
53-
{label: "networking-ingress", fixture: "networking-ingress.yaml", skip: idempotencyReadyBlockedSkip},
54-
{label: "observability-full", fixture: "observability-full.yaml", skip: idempotencyReadyBlockedSkip},
33+
{label: "minimal", fixture: "minimal.yaml"},
34+
{label: "maximal", fixture: "maximal.yaml"},
35+
{label: "gateways-all", fixture: "gateways-all.yaml"},
36+
{label: "selfconfig-enabled", fixture: "selfconfig-enabled.yaml"},
37+
{label: "profilestore-enabled", fixture: "profilestore-enabled.yaml"},
38+
{label: "autoupdate-enabled", fixture: "autoupdate-enabled.yaml"},
39+
{label: "backup-enabled", fixture: "backup-enabled.yaml"},
40+
{label: "networking-ingress", fixture: "networking-ingress.yaml"},
41+
{label: "observability-full", fixture: "observability-full.yaml"},
5542
{
5643
label: "ollama-webterminal-tailscale",
5744
fixture: "ollama-webterminal-tailscale.yaml",

test/conformance/testdata/autoupdate-enabled.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/backup-enabled.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/gateways-all.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/maximal.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ metadata:
77
spec:
88
image:
99
repository: ghcr.io/paperclipinc/hermes-agent
10-
tag: "v2026.5.29.2"
10+
tag: "v0.16.0"
1111
pullPolicy: IfNotPresent
1212
storage:
1313
persistence:

test/conformance/testdata/minimal.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/networking-ingress.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/observability-full.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/ollama-webterminal-tailscale.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

test/conformance/testdata/profilestore-enabled.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ metadata:
55
spec:
66
image:
77
repository: ghcr.io/paperclipinc/hermes-agent
8-
tag: "v2026.5.29.2"
8+
tag: "v0.16.0"
99
storage:
1010
persistence:
1111
enabled: true

0 commit comments

Comments
 (0)