Skip to content

feat!: realign operator with Paperclip's real config contract + fix app-boot blockers - #73

Merged
stubbi merged 21 commits into
mainfrom
feat/config-realignment
Jun 5, 2026
Merged

feat!: realign operator with Paperclip's real config contract + fix app-boot blockers#73
stubbi merged 21 commits into
mainfrom
feat/config-realignment

Conversation

@stubbi

@stubbi stubbi commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Realigns the operator with the Paperclip app's actual configuration contract (verified against paperclipai/paperclip@master), removes config the app never consumed, and fixes the bugs that prevented the app from actually booting on a stock cluster. Verified end-to-end on kind: the real Paperclip app boots to Ready with the realigned config.

Breaking change (feat!:). Existing CRs using spec.redis, spec.adapters.managedInference*, or spec.deployment.mode: open|single-tenant must be migrated before upgrade. See docs/UPGRADING.md.

Config realignment

  • Deployment mode enum open|authenticated|single-tenantlocal_trusted|authenticated (the only values the app accepts) + new spec.auth.disableSignUp for the former single-tenant intent + CEL rule (private exposure when local_trusted). Probe auto-selection updated.
  • Bind: emit PAPERCLIP_BIND=custom + PAPERCLIP_BIND_HOST=0.0.0.0 instead of the legacy HOST.
  • Remove dead Redis (spec.redis + StatefulSet/Service/PVC + NetworkPolicy + env): the app bundles no Redis client and reads no Redis env; it was an idle pod wired to nothing. (Planned to return once the app consumes it.)
  • Remove fabricated managed-inference (spec.adapters.managedInference* / PAPERCLIP_MANAGED_*): these env vars don't exist in the app. Use spec.adapters.apiKeysSecretRef (ANTHROPIC_API_KEY/OPENAI_API_KEY); model discovery is automatic.
  • Add AWS Secrets Manager vault: spec.secrets.provider + spec.secrets.awsPAPERCLIP_SECRETS_PROVIDER / PAPERCLIP_SECRETS_AWS_* (creds via IRSA, no keys injected).
  • Add E2B sandbox key: spec.adapters.e2b.apiKeySecretRefE2B_API_KEY (the only sandbox provider with an env path; Modal/Cloudflare/SSH are UI-configured).
  • Add app-native DB backup: spec.backup.appNativePAPERCLIP_DB_BACKUP_*; spec.backup.schedule is now optional.

App-boot fixes (found via the full-boot e2e — each blocked the app on a stock cluster)

  • Gateway API watch: Owns(&HTTPRoute{}) was unconditional, so without the gateway-api CRDs the controller never started its workers and reconciled nothing. Now watched only when the CRD is present.
  • OTEL preload: NODE_OPTIONS=--import ./server/dist/instrumentation.js was forced always, crashing every container (ERR_MODULE_NOT_FOUND) on images without that file. Gated behind observability.metrics.enabled.
  • gosu entrypoint vs restricted PSS: the image entrypoint drops root→node via gosu, which fails under runAsNonRoot/drop-ALL. The operator now execs node directly (keeping pod-0 heartbeat gating).
  • Authenticated-mode hostname allowlist: always include the in-cluster Service DNS so the operator's own bootstrap (and in-cluster clients) pass the app's allowlist.

Docs / install

  • New docs/deploy/runtime-configured-features.md (MCP stdio server, Modal/Cloudflare/SSH environments, first-admin) and docs/UPGRADING.md migration guide; README/api-reference/samples realigned.
  • Release pipeline now publishes a version-pinned install.yaml asset (fixes the README kubectl apply 404; relates to how to access the container image? #61, which is otherwise resolved).

Test plan

  • make fmt && make vet && make lint
  • make test (unit + envtest integration)
  • make test-e2e on kind: 4 of 4 Specs SUCCESS — real app boots to Ready (managed Postgres, authenticated mode), realigned env asserted, no Redis, feature env (AWS vault / E2B / app-native backup) rendered, clean teardown
  • New unit tests for every builder change + conformance CEL cases

Known follow-ups (not blockers)

  • App registry publishes no semver image tags (only latest/sha-*); operator forbids :latest, so pin a sha-* tag.
  • Bootstrap CEO-promotion is blocked by the app's config-file-vs-env split (onboard --yes writes config.json as local_trusted; the CLI reads the file, not the env the server honors). Admin account creation works; documented in docs/deploy/runtime-configured-features.md.

🤖 Generated with Claude Code

stubbi and others added 21 commits June 4, 2026 11:27
The Paperclip app ships from github.com/paperclipai/paperclip to
ghcr.io/paperclipai/paperclip. The operator defaulted spec.image.repository
to ghcr.io/paperclipinc/paperclip, which is the operator's own vendor org
and not where the app is published, so an Instance without an explicit
repository pulled a non-existent image.

Update the default across the API type, statefulset/controller fallbacks,
generated CRDs, Helm chart templates, OLM bundle, samples, docs, and tests.
The operator's own image (ghcr.io/paperclipinc/paperclip-operator) is
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the invalid open/single-tenant modes (the app only accepts
local_trusted and authenticated). Adds spec.auth.disableSignUp ->
PAPERCLIP_AUTH_DISABLE_SIGN_UP for the no-public-signup case, with a CEL rule
requiring private exposure in local_trusted mode. Probe auto-selection now
uses TCP only for authenticated mode.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Paperclip app bundles no Redis client (no ioredis/redis dependency) and
reads no Redis env var anywhere - server/src/config.ts has no REDIS_URL /
PAPERCLIP_RATE_LIMIT_REDIS_URL, and its only rate limiter is an in-process
in-memory map. The operator's managed Redis StatefulSet/Service/PVC and the
injected PAPERCLIP_RATE_LIMIT_REDIS_URL were therefore inert: an idle pod
connected to nothing.

Removes spec.redis, the Redis builders, the redis NetworkPolicy egress + the
dedicated redis NetworkPolicy, the redis reconcile paths, the RedisReady
condition, the status RedisStatefulSet/Service/PVC fields, and the REDIS_URL
selfconfig allowlist entry.

Horizontal scaling is unaffected: it relies on shared Postgres
(managed/external), object storage (S3/MinIO/R2) for shared files, HPA/PDB/
topology spread, and pod-0 heartbeat gating - none of which used Redis.

Note: Redis is planned to return once the app consumes it, via a custom image
or upstream changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PAPERCLIP_MANAGED_INFERENCE_* / PAPERCLIP_MANAGED_*_API_KEY env vars do not
exist in the Paperclip app (confirmed: zero references in the app source) and
were no-ops. Removes spec.adapters.managedInferenceSecretRef,
managedInferenceProvider, and managedInferenceModel plus the env builder.

LLM keys are configured via spec.adapters.apiKeysSecretRef (ANTHROPIC_API_KEY /
OPENAI_API_KEY); the app discovers models live from the provider.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
spec.secrets.provider (local_encrypted|aws_secrets_manager) and spec.secrets.aws
emit PAPERCLIP_SECRETS_PROVIDER and PAPERCLIP_SECRETS_AWS_* (region, kms key id,
deployment id, prefix, environment, endpoint, delete recovery days). AWS
credentials come from the SDK chain (IRSA via serviceAccountAnnotations); no
keys are injected. CEL requires spec.secrets.aws when the provider is AWS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes spec.backup.schedule optional and adds spec.backup.appNative (enabled,
intervalMinutes, retentionDays) to drive Paperclip's built-in local-dir DB
backups, stored under the data PVC. The operator pg_dump -> S3 CronJob now only
runs when a schedule is set, so app-native backups can be used alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…num, aws secrets)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Regenerate docs/api-reference.md (drops redis/managedInference, adds
  secrets provider, e2b, appNative, disableSignUp).
- README + docs/index: remove Redis and managed-inference, fix deployment-mode
  enum, document AWS secrets vault, E2B key, app-native backup, PAPERCLIP_BIND.
- Add docs/deploy/runtime-configured-features.md (MCP stdio server,
  Modal/Cloudflare/SSH environments, first-admin claim).
- Add docs/UPGRADING.md with the breaking-change migration steps.
- Add commented new-feature examples to the main sample.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an Instance lifecycle Context (nested in the Manager Describe so the
controller stays deployed): a managed-DB authenticated boot to Ready (asserts
corrected env, no Redis, bootstrap Job completes) and a feature-render case
(AWS vault, E2B, app-native backup env). Pins the app image to a pullable
sha tag since the registry publishes no semver image tags. Bumps the e2e
go-test timeout to 45m and dumps namespace diagnostics on failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…talled

SetupWithManager unconditionally called Owns(&gatewayapiv1.HTTPRoute{}). On a
cluster without the Gateway API CRDs (kind, and many real clusters), that
owned-type cache sync fails forever, so the Instance controller never starts
its workers and the operator silently reconciles nothing.

Register the HTTPRoute watch only when gateway.networking.k8s.io/v1 HTTPRoute
is discoverable via the RESTMapper; otherwise log that HTTPRoute support is
disabled. The envtest suite never exercised this (it drives Reconcile directly
without starting the manager), so it surfaced only under a running manager.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three pre-existing defects that prevented the Paperclip app from starting on a
stock cluster, found via the full-boot e2e on kind:

1. OTEL preload forced unconditionally: NODE_OPTIONS=--import
   ./server/dist/instrumentation.js was injected always, but that file only
   exists in images built with instrumentation. On images without it Node
   exits with ERR_MODULE_NOT_FOUND, crashing both the onboard init and the
   main container. Gate the whole OTEL block on observability.metrics.enabled
   (default off), so the app boots by default.

2. Image gosu entrypoint vs restricted securityContext: the app image
   ENTRYPOINT (docker-entrypoint.sh) gosu-drops from root to "node", which
   fails under runAsNonRoot/runAsUser:1000/drop-ALL ("failed switching to
   node: operation not permitted"). The operator only overrode the command in
   the multi-replica case; now always exec the server directly (we already run
   as the node uid), keeping pod-0 heartbeat gating for multi-replica.

3. Authenticated-mode hostname allowlist: the app rejects auth from hostnames
   not in PAPERCLIP_ALLOWED_HOSTNAMES. The operator's own bootstrap Job calls
   the Service DNS, so always include the in-cluster Service names (and
   loopback) in addition to user-specified hostnames.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n follow-up

The boot e2e hard-asserts the operator-scope outcomes (managed Postgres ready,
app Ready, realigned env present, no Redis) and now asserts the bootstrap Job
authenticates to the app (proving the in-cluster hostname allowlist), without
gating on full Job success: the bootstrap-ceo CEO-promotion fails due to the
app config-file vs env split, documented as a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stsEnvNames asserted on the kubectl error via the global Expect, so the
feature-render Eventually hard-failed on its first poll before the e2e-feat
StatefulSet existed. Return '' instead and let the caller's Eventually retry.
(The full app-boot spec already passes; this fixes the feature-render spec.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The README documents `kubectl apply -f .../releases/latest/download/install.yaml`,
but no install.yaml was ever attached to releases (404), and a naive
`kustomize build config/default` references the placeholder `controller:latest`
image. Generate the installer during release with the image pinned to the
released tag and upload it as a release asset (mirroring the SBOM upload).
Also gitignore the generated dist/ output.

Relates to #61 (operator install friction): the image-pull 403 was already
fixed by publishing the public image; this fixes the other documented install
path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n hang

The Manager AfterAll undeploys the operator; the Instance-lifecycle AfterAll only
did a fire-and-forget 'kubectl delete ns --wait=false', so Instance finalizers
were orphaned once the operator was gone, wedging namespace + CRD deletion until
the go-test timeout. Delete Instances and wait for finalizers to clear while the
operator is still running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@stubbi
stubbi merged commit 4b206d7 into main Jun 5, 2026
10 of 14 checks passed
@stubbi
stubbi deleted the feat/config-realignment branch June 5, 2026 03:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant