From ca9c69cea82fdc3ed267770e4b1bc2c06338f076 Mon Sep 17 00:00:00 2001 From: Erick Bourgeois Date: Tue, 23 Jun 2026 11:44:01 +0100 Subject: [PATCH] Make sure binary for the spotSchedule controller is added to 5-spot image Signed-off-by: Erick Bourgeois --- .claude/CHANGELOG.md | 50 +++++++++++++++++++ .../prepare-docker-binaries/action.yaml | 22 +++++++- .github/workflows/build.yaml | 4 ++ Cargo.toml | 13 ++++- Dockerfile | 6 +++ Dockerfile.chainguard | 6 +++ Makefile | 20 +++++--- .../capital-markets/deployment.yaml | 2 +- .../time-based/deployment.yaml | 2 +- docs/src/guides/capital-markets-schedule.md | 8 +++ docs/src/guides/time-based-schedule.md | 14 ++++++ docs/src/installation/controller.md | 9 ++++ 12 files changed, 144 insertions(+), 12 deletions(-) diff --git a/.claude/CHANGELOG.md b/.claude/CHANGELOG.md index 2eb29dc..db2e286 100644 --- a/.claude/CHANGELOG.md +++ b/.claude/CHANGELOG.md @@ -9,6 +9,56 @@ The format is based on the regulated environment requirements: --- +## [2026-06-23 11:45] - Ship the spot-schedule provider binaries in the image (ADR 0009 providers were unrunnable) + +**Author:** Erick Bourgeois + +### Changed +- `Cargo.toml`: added explicit `[[bin]]` entries for `spot-schedule-time-based` + (`src/bin/spot_schedule_time_based.rs`) and `spot-schedule-capital-markets` + (`src/bin/spot_schedule_capital_markets.rs`). They were previously only + auto-discovered under their underscore file-stem names, which did not match the + hyphenated `command:` in their deploy manifests. +- `Dockerfile`, `Dockerfile.chainguard`: also `COPY` `spot-schedule-time-based` and + `spot-schedule-capital-markets` into the image (alongside `/5spot`). The main image + is now multi-binary; `ENTRYPOINT` stays `/5spot` and providers select via `command:`. +- `Makefile` (`prepare-binaries-linux-amd64`, `-arm64`): copy the two provider binaries + into `binaries//` so the Docker build can pick them up. +- `.github/workflows/build.yaml` (Build job): the artifact upload (PR/push + release) + now includes `spot-schedule-time-based` and `spot-schedule-capital-markets` — CI + uploads only the named binary, so without this the providers never reach the Docker + job (the multi-arch build failed with `binaries/arm64/spot-schedule-time-based: not + found`). `cargo build --release` already builds all `[[bin]]`s. +- `.github/actions/prepare-docker-binaries/action.yaml`: copy the two provider binaries + from the downloaded artifacts into `binaries//` for both arches (the local + `make prepare-binaries` is not used by CI — this composite action is). +- `deploy/spot-schedule-providers/{time-based,capital-markets}/deployment.yaml`: + `command` changed to absolute paths (`/spot-schedule-time-based`, + `/spot-schedule-capital-markets`) — the distroless image has no PATH entry for a bare + name; the binaries live at `/`. +- `docs/src/guides/time-based-schedule.md`, `docs/src/guides/capital-markets-schedule.md`, + `docs/src/installation/controller.md`: documented that each provider is a separate, + **required** Deployment shipped inside the main image (selected by `command:`), and + that without a provider running no `ScheduledMachine` referencing a schedule activates. + +### Why +The spot-schedule providers (ADR 0009) had source and deploy manifests but were never +wired into the build: the binaries were not declared with their deployed names, were +not copied by `prepare-binaries`, and were not in the Dockerfiles. The published image +therefore could not run them — a deployed provider crash-looped with +`exec: "spot-schedule-time-based": executable file not found`, leaving every +`ScheduledMachine` stuck (no `status.active` is ever written). Surfaced while bringing +up the k0smotron workshop tier on a v1beta1 image. + +### Impact +- [ ] Breaking change +- [x] Requires cluster rollout +- [ ] Config change only +- [ ] Documentation only (code + docs) + +Note: the next image build is required before the provider Deployments can run; their +`:v0.1.0` placeholder tags are pinned at release by `make set-image-version`. + ## [2026-06-22 12:00] - CAPI Machine API version: passthrough from bootstrapSpec + mandatory discovery (no hardcoded version) **Author:** Erick Bourgeois diff --git a/.github/actions/prepare-docker-binaries/action.yaml b/.github/actions/prepare-docker-binaries/action.yaml index deccf77..4cf1d35 100644 --- a/.github/actions/prepare-docker-binaries/action.yaml +++ b/.github/actions/prepare-docker-binaries/action.yaml @@ -48,6 +48,26 @@ runs: exit 1 fi + # Copy the spot-schedule provider binaries. The main image is multi-binary — + # the provider Deployments select them via `command:` — so the Dockerfile + # COPYs them alongside /5spot. Without this they are absent from the artifacts + # and the build fails ("binaries//spot-schedule-time-based: not found"). + for arch_dir in amd64 arm64; do + art="artifacts/5spot-linux-$arch_dir" + for prov in spot-schedule-time-based spot-schedule-capital-markets; do + PROV_BIN=$(find "$art" -type f -name "$prov" | head -1) + if [ -n "$PROV_BIN" ] && [ -f "$PROV_BIN" ]; then + cp "$PROV_BIN" "binaries/$arch_dir/$prov" + chmod +x "binaries/$arch_dir/$prov" + echo "Copied $prov ($arch_dir) from $PROV_BIN" + else + echo "ERROR: $prov ($arch_dir) not found in $art" + find "$art" -ls || echo "Directory does not exist" + exit 1 + fi + done + done + # Verify binaries - ls -lh binaries/amd64/5spot binaries/arm64/5spot + ls -lh binaries/amd64/* binaries/arm64/* file binaries/amd64/5spot binaries/arm64/5spot diff --git a/.github/workflows/build.yaml b/.github/workflows/build.yaml index b6d735a..e5864f1 100644 --- a/.github/workflows/build.yaml +++ b/.github/workflows/build.yaml @@ -287,6 +287,8 @@ jobs: name: ${{ matrix.platform.artifact_name }} path: | target/release/${{ matrix.platform.binary_name }} + target/release/spot-schedule-time-based + target/release/spot-schedule-capital-markets target/release/*.cdx.json retention-days: 1 @@ -298,6 +300,8 @@ jobs: name: ${{ matrix.platform.artifact_name }} path: | target/release/${{ matrix.platform.binary_name }} + target/release/spot-schedule-time-based + target/release/spot-schedule-capital-markets target/release/*.cdx.json # ───────────────────────────────────────────────────────────────────────────── diff --git a/Cargo.toml b/Cargo.toml index 821c9fe..7a40faa 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "five_spot" -version = "0.1.1" +version = "0.3.0" edition = "2021" rust-version = "1.90" authors = ["Erick Bourgeois"] @@ -28,6 +28,17 @@ path = "src/bin/auto_vex_presence.rs" name = "auto-vex-reachability" path = "src/bin/auto_vex_reachability.rs" +# First-party spot-schedule providers (ADR 0009). Declared explicitly so the binary +# names are hyphenated (matching the `command:` in their deploy manifests) rather than +# the auto-discovered underscore names. Shipped IN the main 5-Spot image. +[[bin]] +name = "spot-schedule-time-based" +path = "src/bin/spot_schedule_time_based.rs" + +[[bin]] +name = "spot-schedule-capital-markets" +path = "src/bin/spot_schedule_capital_markets.rs" + [dependencies] # `unstable-runtime` is opt-in for `Controller::reconcile_on`, which we # use in main.rs to feed per-child-cluster Node-watch events into the diff --git a/Dockerfile b/Dockerfile index f27075d..21b82b9 100644 --- a/Dockerfile +++ b/Dockerfile @@ -37,6 +37,12 @@ LABEL org.opencontainers.image.source="https://github.com/finos/5-spot" \ # Copy the pre-built binary for the target architecture COPY --chmod=755 binaries/${TARGETARCH}/5spot /5spot +# First-party spot-schedule providers (ADR 0009). Shipped in the same image; their +# deploy manifests select them with `command: ["/spot-schedule-"]`. Without +# a provider running, ScheduledMachines never see status.active and never activate. +COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-time-based /spot-schedule-time-based +COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-capital-markets /spot-schedule-capital-markets + USER nonroot EXPOSE 8080 diff --git a/Dockerfile.chainguard b/Dockerfile.chainguard index 925fd01..c31727d 100644 --- a/Dockerfile.chainguard +++ b/Dockerfile.chainguard @@ -48,6 +48,12 @@ LABEL org.opencontainers.image.source="https://github.com/finos/5-spot" \ # TARGETARCH is automatically set by BuildKit (amd64 or arm64) COPY --chmod=755 binaries/${TARGETARCH}/5spot /5spot +# First-party spot-schedule providers (ADR 0009). Shipped in the same image; their +# deploy manifests select them with `command: ["/spot-schedule-"]`. Without +# a provider running, ScheduledMachines never see status.active and never activate. +COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-time-based /spot-schedule-time-based +COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-capital-markets /spot-schedule-capital-markets + USER 65532:65532 EXPOSE 8080 diff --git a/Makefile b/Makefile index 6f9dbb7..5204570 100644 --- a/Makefile +++ b/Makefile @@ -134,19 +134,23 @@ build-linux-arm64: ## Build for Linux ARM64 (requires cross toolchain) exit 1; \ fi -prepare-binaries-linux-amd64: build-linux-amd64 ## Build and prepare Linux x86_64 binary - @echo "Preparing Linux x86_64 binary for Docker build..." +prepare-binaries-linux-amd64: build-linux-amd64 ## Build and prepare Linux x86_64 binaries + @echo "Preparing Linux x86_64 binaries for Docker build..." @mkdir -p binaries/amd64 @cp target/x86_64-unknown-linux-gnu/release/5spot binaries/amd64/ - @echo "✓ Binary ready: binaries/amd64/5spot" - @ls -lh binaries/amd64/5spot + @cp target/x86_64-unknown-linux-gnu/release/spot-schedule-time-based binaries/amd64/ + @cp target/x86_64-unknown-linux-gnu/release/spot-schedule-capital-markets binaries/amd64/ + @echo "✓ Binaries ready: 5spot, spot-schedule-time-based, spot-schedule-capital-markets" + @ls -lh binaries/amd64/ -prepare-binaries-linux-arm64: build-linux-arm64 ## Build and prepare Linux ARM64 binary - @echo "Preparing Linux ARM64 binary for Docker build..." +prepare-binaries-linux-arm64: build-linux-arm64 ## Build and prepare Linux ARM64 binaries + @echo "Preparing Linux ARM64 binaries for Docker build..." @mkdir -p binaries/arm64 @cp target/aarch64-unknown-linux-gnu/release/5spot binaries/arm64/ - @echo "✓ Binary ready: binaries/arm64/5spot" - @ls -lh binaries/arm64/5spot + @cp target/aarch64-unknown-linux-gnu/release/spot-schedule-time-based binaries/arm64/ + @cp target/aarch64-unknown-linux-gnu/release/spot-schedule-capital-markets binaries/arm64/ + @echo "✓ Binaries ready: 5spot, spot-schedule-time-based, spot-schedule-capital-markets" + @ls -lh binaries/arm64/ test: ## Run all tests cargo test --all diff --git a/deploy/spot-schedule-providers/capital-markets/deployment.yaml b/deploy/spot-schedule-providers/capital-markets/deployment.yaml index b99aa33..679da10 100644 --- a/deploy/spot-schedule-providers/capital-markets/deployment.yaml +++ b/deploy/spot-schedule-providers/capital-markets/deployment.yaml @@ -41,7 +41,7 @@ spec: # Pin to a released version tag — NEVER use :latest (Trivy KSV-0013). image: ghcr.io/finos/5-spot:v0.1.0 imagePullPolicy: IfNotPresent - command: ["spot-schedule-capital-markets"] + command: ["/spot-schedule-capital-markets"] env: - name: RUST_LOG value: "info" diff --git a/deploy/spot-schedule-providers/time-based/deployment.yaml b/deploy/spot-schedule-providers/time-based/deployment.yaml index db08ad7..86bd104 100644 --- a/deploy/spot-schedule-providers/time-based/deployment.yaml +++ b/deploy/spot-schedule-providers/time-based/deployment.yaml @@ -41,7 +41,7 @@ spec: # Pin to a released version tag — NEVER use :latest (Trivy KSV-0013). image: ghcr.io/finos/5-spot:v0.1.0 imagePullPolicy: IfNotPresent - command: ["spot-schedule-time-based"] + command: ["/spot-schedule-time-based"] env: - name: RUST_LOG value: "info" diff --git a/docs/src/guides/capital-markets-schedule.md b/docs/src/guides/capital-markets-schedule.md index 7237a96..6d24c43 100644 --- a/docs/src/guides/capital-markets-schedule.md +++ b/docs/src/guides/capital-markets-schedule.md @@ -43,6 +43,14 @@ kubectl apply -f deploy/crds/capitalmarketsschedule.yaml kubectl apply -k deploy/spot-schedule-providers/capital-markets/ ``` +!!! important "The provider is a separate, required Deployment" + Like every spot-schedule provider, this is its **own controller** + (`spot-schedule-capital-markets`) — the only writer of + `CapitalMarketsSchedule.status.active`. If it is not running, no + `ScheduledMachine` referencing a `CapitalMarketsSchedule` activates. The binary + ships **inside the main 5-Spot image** (multi-binary); the Deployment uses the + `ghcr.io/finos/5-spot` image with `command: ["/spot-schedule-capital-markets"]`. + The provider runs with a least-privilege ClusterRole: `get;list;watch` on `capitalmarketsschedules` and `update;patch` on **only** their `/status` subresource — it never writes the spec (operators own the calendar) and reads diff --git a/docs/src/guides/time-based-schedule.md b/docs/src/guides/time-based-schedule.md index 92f0f9e..d536f47 100644 --- a/docs/src/guides/time-based-schedule.md +++ b/docs/src/guides/time-based-schedule.md @@ -46,6 +46,20 @@ kubectl apply -f deploy/crds/timebasedspotschedule.yaml kubectl apply -k deploy/spot-schedule-providers/time-based/ ``` +!!! important "The provider is a separate Deployment — and it is required" + The provider is its **own controller** (`spot-schedule-time-based`), distinct + from the 5-Spot controller. It is the only thing that writes + `TimeBasedSpotSchedule.status.active`, so **if it is not running, no + `ScheduledMachine` that references a `TimeBasedSpotSchedule` ever activates** + (5-Spot treats an absent/`StatusActiveMissing` verdict as unresolved). It runs + as a `Deployment` in `5spot-system`, separate from `deploy/deployment/`. + + The provider binary ships **inside the main 5-Spot image** — the Deployment uses + the same `ghcr.io/finos/5-spot` image as the controller and selects the provider + with `command: ["/spot-schedule-time-based"]` (the image is multi-binary: `/5spot` + plus each first-party provider). `make set-image-version VERSION=…` pins its tag + alongside the controller at release. + The provider runs with a least-privilege ClusterRole: `get;list;watch` on `timebasedspotschedules` and `update;patch` on **only** their `/status` subresource — it never writes the spec (operators own the window) and reads diff --git a/docs/src/installation/controller.md b/docs/src/installation/controller.md index 0ba3f61..8c5fc15 100644 --- a/docs/src/installation/controller.md +++ b/docs/src/installation/controller.md @@ -15,6 +15,15 @@ The `-R` (recursive) flag is required so `deploy/deployment/rbac/` it the Deployment will be created but fail to schedule with `serviceaccount "5spot-controller" not found`. +!!! important "You also need a spot-schedule provider" + The controller alone does not decide *when* a machine is active — it reads + `status.active` from a spot-schedule **provider** (ADR 0009). `deploy/deployment/` + does **not** include one, so install at least the first-party + [`TimeBasedSpotSchedule` provider](../guides/time-based-schedule.md#install) + (`kubectl apply -k deploy/spot-schedule-providers/time-based/`). Without a provider + running, `ScheduledMachine`s referencing a schedule never activate. The provider + binaries ship inside the same `ghcr.io/finos/5-spot` image (selected by `command:`). + ### Using Helm (Coming Soon) ```bash