Skip to content

Provider production-hardening: deployable, correct, tested - #1

Merged
stubbi merged 29 commits into
mainfrom
feat/provider-production-hardening
Jun 13, 2026
Merged

Provider production-hardening: deployable, correct, tested#1
stubbi merged 29 commits into
mainfrom
feat/provider-production-hardening

Conversation

@stubbi

@stubbi stubbi commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes karpenter-provider-hetzner actually deployable and production-correct (Spec #1 of 4). Design + plan: docs/superpowers/specs/2026-06-13-provider-production-hardening-design.md.

Highlights:

  • Build/CI fixed. Go 1.26 aligned across Dockerfile/CI/release; CI routes through make. CRD is generated, committed, and guarded by a CI drift check (make generate-verify).
  • NodeClass status controller (new pkg/controllers/nodeclass): validates network, firewalls, and SSH keys exist and resolves the image family per architecture; sets ImagesReady/NetworkReady/ResourcesReady aggregated into Ready (which Karpenter gates provisioning on); populates arch-keyed Status.ResolvedImages; periodic resync.
  • Robust instance.Create: waits on create actions before returning; maps Hetzner resource_unavailable/resource_limit_exceeded to Karpenter InsufficientCapacityError so the scheduler falls back to another type/zone.
  • Unavailable-offerings cache: a capacity failure marks (serverType, location) unavailable for a TTL, applied live per List without staleness-trapping the 6h type catalog.
  • Pricing + cost control: switched to Net hourly pricing; added enablePublicIPv4/enablePublicIPv6 NodeClass flags (default true) so private-network clusters can drop the billed primary IPv4.
  • Complete drift detection: image, network, firewall, server-type. SSH-key/user-data drift intentionally omitted (Hetzner can't reliably read them) and documented rather than faked.
  • Multi-cluster safety: required CLUSTER_NAME (charset-validated), tagged on every server, and filtered in List/GC so two clusters in one Hetzner project never collide.
  • Helm hardening: injects CLUSTER_NAME (fail-fast if unset), metrics/health ports, and liveness/readiness probes.

Critical bug fixed

The provider stamped custom requirement labels (karpenter.hetzner.cloud/cpu-type, /server-family) on instance types but never registered them in Karpenter's WellKnownLabels — so Create rejected every instance type for any normal NodeClaim, making provisioning impossible. Surfaced by the new CloudProvider tests; fixed via pkg/apis/v1alpha1/wellknown.go.

Test Plan

  • go build ./...
  • go test -race -count=1 ./... (all 7 packages green)
  • make generate-verify (committed CRD matches types)
  • helm lint + fail-fast on missing clusterName
  • golangci-lint run ./... — runs in CI lint job (not installed locally)
  • End-to-end provisioning on the mono Hetzner/Talos cluster with cost guardrails — Spec Repo polish: README, contributor docs, issue/PR templates, security tooling #2 (separate)

Scope / follow-ups (own specs)

🤖 Generated with Claude Code

stubbi and others added 29 commits June 13, 2026 10:44
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-true

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds LoadConfig() reading CLUSTER_NAME from the environment (required),
threads the cluster name into instance.Provider, tags every created
server with karpenter.sh/cluster=<name>, and filters List() by both
the managed-by and cluster labels so two clusters in one Hetzner
project cannot see each other's servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eate tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ches

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… harness

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…set + label-skip

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without this, CloudProvider.Create rejected every instance type because the
custom cpu-type/server-family requirement labels were undefined, breaking all
provisioning. Surfaced by cloudprovider Create tests.
…e Create

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r config

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
golangci-lint v1.64.8 (built with go1.24) refused to lint a module targeting
go1.26.3. Upgrade to golangci-lint v2 via action@v8 with a v2-format config
(gosimple merged into staticcheck), and let go mod tidy settle the go directive
to the dependency floor (1.26.2) instead of the spurious 1.26.3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- register HCloudNodeClass into the client-go global scheme in init() so the
  operator manager (built from scheme.Scheme) can watch it at runtime; this also
  replaces the deprecated controller-runtime scheme.Builder (SA1019)
- applyAvailability: construct a fresh InstanceType instead of copying *it,
  fixing govet copylocks (InstanceType embeds sync.Once)
- drop unused abs() test helper (unused); simplify promoted-field selector (QF1008)
- gofmt

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@stubbi
stubbi merged commit 579fba4 into main Jun 13, 2026
4 checks passed
@stubbi
stubbi deleted the feat/provider-production-hardening branch June 13, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant