Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .claude/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,56 @@ The format is based on the regulated environment requirements:

---

## [2026-06-23 11:45] - Ship the spot-schedule provider binaries in the image (ADR 0009 providers were unrunnable)

**Author:** Erick Bourgeois

### Changed
- `Cargo.toml`: added explicit `[[bin]]` entries for `spot-schedule-time-based`
(`src/bin/spot_schedule_time_based.rs`) and `spot-schedule-capital-markets`
(`src/bin/spot_schedule_capital_markets.rs`). They were previously only
auto-discovered under their underscore file-stem names, which did not match the
hyphenated `command:` in their deploy manifests.
- `Dockerfile`, `Dockerfile.chainguard`: also `COPY` `spot-schedule-time-based` and
`spot-schedule-capital-markets` into the image (alongside `/5spot`). The main image
is now multi-binary; `ENTRYPOINT` stays `/5spot` and providers select via `command:`.
- `Makefile` (`prepare-binaries-linux-amd64`, `-arm64`): copy the two provider binaries
into `binaries/<arch>/` so the Docker build can pick them up.
- `.github/workflows/build.yaml` (Build job): the artifact upload (PR/push + release)
now includes `spot-schedule-time-based` and `spot-schedule-capital-markets` — CI
uploads only the named binary, so without this the providers never reach the Docker
job (the multi-arch build failed with `binaries/arm64/spot-schedule-time-based: not
found`). `cargo build --release` already builds all `[[bin]]`s.
- `.github/actions/prepare-docker-binaries/action.yaml`: copy the two provider binaries
from the downloaded artifacts into `binaries/<arch>/` for both arches (the local
`make prepare-binaries` is not used by CI — this composite action is).
- `deploy/spot-schedule-providers/{time-based,capital-markets}/deployment.yaml`:
`command` changed to absolute paths (`/spot-schedule-time-based`,
`/spot-schedule-capital-markets`) — the distroless image has no PATH entry for a bare
name; the binaries live at `/`.
- `docs/src/guides/time-based-schedule.md`, `docs/src/guides/capital-markets-schedule.md`,
`docs/src/installation/controller.md`: documented that each provider is a separate,
**required** Deployment shipped inside the main image (selected by `command:`), and
that without a provider running no `ScheduledMachine` referencing a schedule activates.

### Why
The spot-schedule providers (ADR 0009) had source and deploy manifests but were never
wired into the build: the binaries were not declared with their deployed names, were
not copied by `prepare-binaries`, and were not in the Dockerfiles. The published image
therefore could not run them — a deployed provider crash-looped with
`exec: "spot-schedule-time-based": executable file not found`, leaving every
`ScheduledMachine` stuck (no `status.active` is ever written). Surfaced while bringing
up the k0smotron workshop tier on a v1beta1 image.

### Impact
- [ ] Breaking change
- [x] Requires cluster rollout
- [ ] Config change only
- [ ] Documentation only (code + docs)

Note: the next image build is required before the provider Deployments can run; their
`:v0.1.0` placeholder tags are pinned at release by `make set-image-version`.

## [2026-06-22 12:00] - CAPI Machine API version: passthrough from bootstrapSpec + mandatory discovery (no hardcoded version)

**Author:** Erick Bourgeois
Expand Down
22 changes: 21 additions & 1 deletion .github/actions/prepare-docker-binaries/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,26 @@ runs:
exit 1
fi

# Copy the spot-schedule provider binaries. The main image is multi-binary —
# the provider Deployments select them via `command:` — so the Dockerfile
# COPYs them alongside /5spot. Without this they are absent from the artifacts
# and the build fails ("binaries/<arch>/spot-schedule-time-based: not found").
for arch_dir in amd64 arm64; do
art="artifacts/5spot-linux-$arch_dir"
for prov in spot-schedule-time-based spot-schedule-capital-markets; do
PROV_BIN=$(find "$art" -type f -name "$prov" | head -1)
if [ -n "$PROV_BIN" ] && [ -f "$PROV_BIN" ]; then
cp "$PROV_BIN" "binaries/$arch_dir/$prov"
chmod +x "binaries/$arch_dir/$prov"
echo "Copied $prov ($arch_dir) from $PROV_BIN"
else
echo "ERROR: $prov ($arch_dir) not found in $art"
find "$art" -ls || echo "Directory does not exist"
exit 1
fi
done
done

# Verify binaries
ls -lh binaries/amd64/5spot binaries/arm64/5spot
ls -lh binaries/amd64/* binaries/arm64/*
file binaries/amd64/5spot binaries/arm64/5spot
4 changes: 4 additions & 0 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,8 @@ jobs:
name: ${{ matrix.platform.artifact_name }}
path: |
target/release/${{ matrix.platform.binary_name }}
target/release/spot-schedule-time-based
target/release/spot-schedule-capital-markets
target/release/*.cdx.json
retention-days: 1

Expand All @@ -298,6 +300,8 @@ jobs:
name: ${{ matrix.platform.artifact_name }}
path: |
target/release/${{ matrix.platform.binary_name }}
target/release/spot-schedule-time-based
target/release/spot-schedule-capital-markets
target/release/*.cdx.json

# ─────────────────────────────────────────────────────────────────────────────
Expand Down
13 changes: 12 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "five_spot"
version = "0.1.1"
version = "0.3.0"
edition = "2021"
rust-version = "1.90"
authors = ["Erick Bourgeois<erick.bourgeois@rbccm.com>"]
Expand Down Expand Up @@ -28,6 +28,17 @@ path = "src/bin/auto_vex_presence.rs"
name = "auto-vex-reachability"
path = "src/bin/auto_vex_reachability.rs"

# First-party spot-schedule providers (ADR 0009). Declared explicitly so the binary
# names are hyphenated (matching the `command:` in their deploy manifests) rather than
# the auto-discovered underscore names. Shipped IN the main 5-Spot image.
[[bin]]
name = "spot-schedule-time-based"
path = "src/bin/spot_schedule_time_based.rs"

[[bin]]
name = "spot-schedule-capital-markets"
path = "src/bin/spot_schedule_capital_markets.rs"

[dependencies]
# `unstable-runtime` is opt-in for `Controller::reconcile_on`, which we
# use in main.rs to feed per-child-cluster Node-watch events into the
Expand Down
6 changes: 6 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@ LABEL org.opencontainers.image.source="https://github.com/finos/5-spot" \
# Copy the pre-built binary for the target architecture
COPY --chmod=755 binaries/${TARGETARCH}/5spot /5spot

# First-party spot-schedule providers (ADR 0009). Shipped in the same image; their
# deploy manifests select them with `command: ["/spot-schedule-<provider>"]`. Without
# a provider running, ScheduledMachines never see status.active and never activate.
COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-time-based /spot-schedule-time-based
COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-capital-markets /spot-schedule-capital-markets

USER nonroot

EXPOSE 8080
Expand Down
6 changes: 6 additions & 0 deletions Dockerfile.chainguard
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
ARG VERSION
ARG GIT_SHA

LABEL org.opencontainers.image.source="https://github.com/finos/5-spot" \

Check warning on line 40 in Dockerfile.chainguard

View workflow job for this annotation

GitHub Actions / 🐳 Build and Push Docker Image - Chainguard

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$BASE_IMAGE' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/
org.opencontainers.image.description="5-Spot Machine Scheduler - Kubernetes Controller for Time-Based Machine Scheduling (Chainguard - Zero CVEs)" \
org.opencontainers.image.licenses="MIT" \
org.opencontainers.image.version="${VERSION}" \
Expand All @@ -48,6 +48,12 @@
# TARGETARCH is automatically set by BuildKit (amd64 or arm64)
COPY --chmod=755 binaries/${TARGETARCH}/5spot /5spot

# First-party spot-schedule providers (ADR 0009). Shipped in the same image; their
# deploy manifests select them with `command: ["/spot-schedule-<provider>"]`. Without
# a provider running, ScheduledMachines never see status.active and never activate.
COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-time-based /spot-schedule-time-based
COPY --chmod=755 binaries/${TARGETARCH}/spot-schedule-capital-markets /spot-schedule-capital-markets

USER 65532:65532

EXPOSE 8080
Expand Down
20 changes: 12 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -134,19 +134,23 @@ build-linux-arm64: ## Build for Linux ARM64 (requires cross toolchain)
exit 1; \
fi

prepare-binaries-linux-amd64: build-linux-amd64 ## Build and prepare Linux x86_64 binary
@echo "Preparing Linux x86_64 binary for Docker build..."
prepare-binaries-linux-amd64: build-linux-amd64 ## Build and prepare Linux x86_64 binaries
@echo "Preparing Linux x86_64 binaries for Docker build..."
@mkdir -p binaries/amd64
@cp target/x86_64-unknown-linux-gnu/release/5spot binaries/amd64/
@echo "✓ Binary ready: binaries/amd64/5spot"
@ls -lh binaries/amd64/5spot
@cp target/x86_64-unknown-linux-gnu/release/spot-schedule-time-based binaries/amd64/
@cp target/x86_64-unknown-linux-gnu/release/spot-schedule-capital-markets binaries/amd64/
@echo "✓ Binaries ready: 5spot, spot-schedule-time-based, spot-schedule-capital-markets"
@ls -lh binaries/amd64/

prepare-binaries-linux-arm64: build-linux-arm64 ## Build and prepare Linux ARM64 binary
@echo "Preparing Linux ARM64 binary for Docker build..."
prepare-binaries-linux-arm64: build-linux-arm64 ## Build and prepare Linux ARM64 binaries
@echo "Preparing Linux ARM64 binaries for Docker build..."
@mkdir -p binaries/arm64
@cp target/aarch64-unknown-linux-gnu/release/5spot binaries/arm64/
@echo "✓ Binary ready: binaries/arm64/5spot"
@ls -lh binaries/arm64/5spot
@cp target/aarch64-unknown-linux-gnu/release/spot-schedule-time-based binaries/arm64/
@cp target/aarch64-unknown-linux-gnu/release/spot-schedule-capital-markets binaries/arm64/
@echo "✓ Binaries ready: 5spot, spot-schedule-time-based, spot-schedule-capital-markets"
@ls -lh binaries/arm64/

test: ## Run all tests
cargo test --all
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
# Pin to a released version tag — NEVER use :latest (Trivy KSV-0013).
image: ghcr.io/finos/5-spot:v0.1.0
imagePullPolicy: IfNotPresent
command: ["spot-schedule-capital-markets"]
command: ["/spot-schedule-capital-markets"]
env:
- name: RUST_LOG
value: "info"
Expand Down
2 changes: 1 addition & 1 deletion deploy/spot-schedule-providers/time-based/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ spec:
# Pin to a released version tag — NEVER use :latest (Trivy KSV-0013).
image: ghcr.io/finos/5-spot:v0.1.0
imagePullPolicy: IfNotPresent
command: ["spot-schedule-time-based"]
command: ["/spot-schedule-time-based"]
env:
- name: RUST_LOG
value: "info"
Expand Down
8 changes: 8 additions & 0 deletions docs/src/guides/capital-markets-schedule.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,14 @@ kubectl apply -f deploy/crds/capitalmarketsschedule.yaml
kubectl apply -k deploy/spot-schedule-providers/capital-markets/
```

!!! important "The provider is a separate, required Deployment"
Like every spot-schedule provider, this is its **own controller**
(`spot-schedule-capital-markets`) — the only writer of
`CapitalMarketsSchedule.status.active`. If it is not running, no
`ScheduledMachine` referencing a `CapitalMarketsSchedule` activates. The binary
ships **inside the main 5-Spot image** (multi-binary); the Deployment uses the
`ghcr.io/finos/5-spot` image with `command: ["/spot-schedule-capital-markets"]`.

The provider runs with a least-privilege ClusterRole: `get;list;watch` on
`capitalmarketsschedules` and `update;patch` on **only** their `/status`
subresource — it never writes the spec (operators own the calendar) and reads
Expand Down
14 changes: 14 additions & 0 deletions docs/src/guides/time-based-schedule.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ kubectl apply -f deploy/crds/timebasedspotschedule.yaml
kubectl apply -k deploy/spot-schedule-providers/time-based/
```

!!! important "The provider is a separate Deployment — and it is required"
The provider is its **own controller** (`spot-schedule-time-based`), distinct
from the 5-Spot controller. It is the only thing that writes
`TimeBasedSpotSchedule.status.active`, so **if it is not running, no
`ScheduledMachine` that references a `TimeBasedSpotSchedule` ever activates**
(5-Spot treats an absent/`StatusActiveMissing` verdict as unresolved). It runs
as a `Deployment` in `5spot-system`, separate from `deploy/deployment/`.

The provider binary ships **inside the main 5-Spot image** — the Deployment uses
the same `ghcr.io/finos/5-spot` image as the controller and selects the provider
with `command: ["/spot-schedule-time-based"]` (the image is multi-binary: `/5spot`
plus each first-party provider). `make set-image-version VERSION=…` pins its tag
alongside the controller at release.

The provider runs with a least-privilege ClusterRole: `get;list;watch` on
`timebasedspotschedules` and `update;patch` on **only** their `/status`
subresource — it never writes the spec (operators own the window) and reads
Expand Down
9 changes: 9 additions & 0 deletions docs/src/installation/controller.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,15 @@ The `-R` (recursive) flag is required so `deploy/deployment/rbac/`
it the Deployment will be created but fail to schedule with
`serviceaccount "5spot-controller" not found`.

!!! important "You also need a spot-schedule provider"
The controller alone does not decide *when* a machine is active — it reads
`status.active` from a spot-schedule **provider** (ADR 0009). `deploy/deployment/`
does **not** include one, so install at least the first-party
[`TimeBasedSpotSchedule` provider](../guides/time-based-schedule.md#install)
(`kubectl apply -k deploy/spot-schedule-providers/time-based/`). Without a provider
running, `ScheduledMachine`s referencing a schedule never activate. The provider
binaries ship inside the same `ghcr.io/finos/5-spot` image (selected by `command:`).

### Using Helm (Coming Soon)

```bash
Expand Down
Loading