Skip to content

Commit d35ece0

Browse files
committed
SpotSchedule: Add new 'capital markets' provider and remaining work for docs
Signed-off-by: Erick Bourgeois <erick@jeb.ca>
1 parent 883db3b commit d35ece0

32 files changed

Lines changed: 1662 additions & 20 deletions

.claude/CHANGELOG.md

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,274 @@ The format is based on the regulated environment requirements:
99

1010
---
1111

12+
## [2026-06-15 10:00] - Spot-schedule docs: contract finalization + concept page (Phase 6)
13+
14+
**Author:** Erick Bourgeois
15+
16+
### Changed
17+
- `docs/src/reference/spot-schedule-contract.md`: finalized — resolved the
18+
`Ready` ambiguity (present non-`True` ⇒ unresolved; **absent ⇒ active
19+
authoritative**), removed the open TODOs, documented exactly which status
20+
fields 5-Spot keys off, and added a worked **"implement your own provider"**
21+
walkthrough (a minimal `ManualSchedule` hello-world) plus links to the
22+
reference provider guide/source. Status banner → stable/Accepted.
23+
- `docs/src/concepts/spot-schedule.md`: NEW concept page — why duck-typing over
24+
more `spec.schedule` fields, referencing a provider, AND composition with
25+
`killSwitch` precedence, the event-driven watch, and the fail-safe
26+
hold-last-state behavior. Added to `docs/mkdocs.yml` nav.
27+
- `docs/src/concepts/scheduled-machine.md`: documented the optional `schedule`
28+
and the new `spotSchedule` reference field (+ AND composition note).
29+
- `docs/adr/0006-...md`: §4 reworded to match the §2 table — resolved the
30+
internal `Ready=False` vs `False/missing` inconsistency (absent ⇒ authoritative).
31+
32+
### Why
33+
Phase 6 (final phase): make the contract implementable from the docs alone — a
34+
provider author can read the contract page + concept page and ship a conformant
35+
provider without reading 5-Spot's source.
36+
37+
### Impact
38+
- [ ] Breaking change
39+
- [ ] Requires cluster rollout
40+
- [ ] Config change only
41+
- [x] Documentation only
42+
43+
---
44+
45+
## [2026-06-14 17:30] - CapitalMarketsSchedule reference provider controller (Phase 5)
46+
47+
**Author:** Erick Bourgeois
48+
49+
### Changed
50+
- `src/providers/capital_markets.rs` (+ `_tests.rs`): NEW reference spot-schedule
51+
provider. Pure calendar logic — `is_active_at` (holiday → closed; else must be
52+
in a `sessions[*]` day+hour window, reusing `parse_day_ranges`/`parse_hour_ranges`;
53+
`earlyCloses` truncates the day) and `next_transition` (hour-by-hour scan to the
54+
next flip, bounded by `PROVIDER_TRANSITION_HORIZON_HOURS`). `reconcile` patches
55+
`status.active`/`observedGeneration`/`lastTransitionTime`/`nextTransitionTime`
56+
+ a `Ready` condition (reason `SessionOpen`/`SessionClosed`) and **requeues
57+
once at the next calendar boundary** (event-driven single timed requeue, no
58+
polling, no network calls). `run()` builds the `Controller`.
59+
- `src/providers/mod.rs`, `src/lib.rs`: new `providers` module.
60+
- `src/bin/spot_schedule_capital_markets.rs`: NEW thin controller binary
61+
(tracing + client + `/metrics` server + `capital_markets::run`).
62+
- `src/constants.rs`: `PROVIDER_TRANSITION_HORIZON_HOURS`,
63+
`PROVIDER_FALLBACK_REQUEUE_SECS`, `PROVIDER_ERROR_REQUEUE_SECS`,
64+
`REASON_CAPITAL_MARKETS_SESSION_OPEN/CLOSED`.
65+
- `src/metrics.rs`: `fivespot_capital_markets_active{namespace,name}` gauge +
66+
`fivespot_capital_markets_transitions_total{namespace,name}` counter (+ record fns).
67+
- `deploy/spot-schedule-providers/capital-markets/`: NEW kustomization —
68+
ServiceAccount, least-privilege ClusterRole (`get/list/watch` on
69+
`capitalmarketsschedules`, `update/patch` on **only** `/status`) + binding, and
70+
a hardened Deployment (non-root, RO-rootfs, caps dropped, seccomp). `trivy
71+
config` clean (0 findings).
72+
- Tests: 13 calendar tests (mid-session/pre-open/post-close/weekend/holiday/
73+
early-close, invalid tz, no-sessions; next-transition open→close, close→open,
74+
weekend-skip, none). 627 lib tests green.
75+
- Docs: `docs/src/guides/capital-markets-schedule.md` (+ nav),
76+
`docs/src/operations/monitoring.md` provider metrics + alert.
77+
78+
### Why
79+
Phase 5 of the spot-schedule roadmap: the reference provider that makes the
80+
Phase 1 `CapitalMarketsSchedule` type *do* something — computing `status.active`
81+
from a declarative exchange calendar so a `ScheduledMachine.spec.spotSchedule`
82+
can follow market sessions. Producer side; independent of the consumer-side
83+
resolver/watch.
84+
85+
### Impact
86+
- [ ] Breaking change
87+
- [x] Requires cluster rollout (apply `deploy/spot-schedule-providers/capital-markets/`)
88+
- [ ] Config change only
89+
- [ ] Documentation only
90+
91+
---
92+
93+
<<<<<<< Updated upstream
94+
=======
95+
## [2026-06-14 16:00] - Auto-VEX pre-submission sign-off gate (ADR 0008)
96+
97+
**Author:** Erick Bourgeois
98+
99+
### Changed
100+
- `docs/adr/0008-autovex-presubmission-gate.md`: NEW ADR — commit a frozen
101+
snapshot of auto-VEX inputs + canonical output; enforce with a byte-exact CI
102+
gate (PR hard-fail). No CALM impact (CI/CD policy). Index updated in
103+
`docs/adr/README.md`.
104+
- `Makefile`: NEW `vex-auto` (regenerate `.vex/auto/*` from `.vex/snapshot/` with
105+
pinned `--id`/`--author`/`--timestamp`) and `vex-auto-check`
106+
(`vex-auto` + `git diff --exit-code -- .vex/auto`) targets; added both to
107+
`.PHONY`.
108+
- `.vex/snapshot/`: NEW frozen generator inputs (`grype.json`, `sbom-bootstrap.json`,
109+
`symbols.txt`, `timestamp.txt`) + `README.md` documenting the sign-off contract
110+
and how to refresh from a real CI capture. Seeded as a valid empty bootstrap.
111+
- `.vex/auto/`: NEW committed canonical output (`vex.auto-presence.json`,
112+
`vex.auto-reachability.json`) — the human-signed-off baseline.
113+
- `.github/workflows/build.yaml`: NEW `validate-autovex` job — runs
114+
`make vex-auto-check` on `pull_request` + `push`, hard-failing when committed
115+
auto-VEX is stale.
116+
- `docs/src/security/vex.md`: NEW "Auto-VEX is signed off before submission"
117+
section documenting the committed baseline, workflow, and gate scope.
118+
119+
### Why
120+
Auto-VEX statements are machine-authored suppressions that previously appeared
121+
only in CI artifacts and were merged into the signed release VEX with no human
122+
review until the Security team's post-hoc counter-signature. This adds an
123+
**upstream** checkpoint: maintainers run the generators, review the suppressions,
124+
and commit them (sign-off) before submission; CI refuses any change whose
125+
committed auto-VEX no longer matches a fresh, deterministic regeneration.
126+
127+
### Impact
128+
- [ ] Breaking change
129+
- [ ] Requires cluster rollout
130+
- [x] Config change only (CI gate + repo convention)
131+
- [ ] Documentation only
132+
133+
---
134+
135+
## [2026-06-14 14:30] - Spot-schedule RBAC + admission + threat model (Phase 4)
136+
137+
**Author:** Erick Bourgeois
138+
139+
### Changed
140+
- `deploy/deployment/rbac/clusterrole.yaml`: NEW rule — `get;list;watch` on
141+
`spotschedules.5spot.finos.org` `*` (read-only, no write). Unblocks the Phase 3
142+
provider watch; least-privilege per ADR 0006 §6 (providers are untrusted
143+
inputs the controller never writes).
144+
- `deploy/admission/validatingadmissionpolicy.yaml`: **security fix**
145+
`matchConstraints.apiVersions` now covers `v1beta1` as well as `v1alpha1`.
146+
Since Phase 1 made `v1beta1` the served/storage version, every `v1beta1`
147+
ScheduledMachine (i.e. every `spotSchedule` machine) was **bypassing the
148+
entire VAP** — the bootstrap/infra allowlist and privilege-escalation guards
149+
included. Also: the four schedule rules are now guarded with
150+
`!has(object.spec.schedule) || …` (schedule is optional since v1beta1, so a
151+
valid spotSchedule-only machine is no longer wrongly rejected), and NEW rules
152+
7a (at least one of schedule/spotSchedule) + 7b (spotSchedule group pin)
153+
mirror the CRD CEL and `validate_activation_source()`.
154+
- `docs/src/security/threat-model.md`: NEW actor (spot-schedule provider,
155+
untrusted), threat T8a (provider-driven activation control — mitigated:
156+
read-only RBAC, same-namespace, group-pinned, auditable) and D5 (provider
157+
flapping — partially mitigated: back-off + lifecycle timers + transition
158+
metric/alert).
159+
- `docs/src/security/admission-validation.md`: documented the dual-version
160+
match, the optional-schedule guard, and rules 7a/7b.
161+
- `docs/src/operations/monitoring.md`: suggested `SpotScheduleProviderFlapping`
162+
and `SpotScheduleProviderUnresolved` Prometheus alerts.
163+
- `docs/architecture/calm/architecture.json`: `rel-controller-spot-schedule-watch`
164+
least-privilege control evidence now cites the implemented ClusterRole + VAP.
165+
`make calm-validate` clean.
166+
167+
### Why
168+
Phase 4: realize ADR 0006 §6's least-privilege, providers-are-untrusted posture
169+
in the deployed manifests, and bring the threat model current. The VAP
170+
version-match fix also closes an admission-bypass regression that Phase 1's
171+
v1beta1 bump introduced.
172+
173+
### Impact
174+
- [ ] Breaking change
175+
- [x] Requires cluster rollout (re-apply ClusterRole + ValidatingAdmissionPolicy)
176+
- [ ] Config change only
177+
- [ ] Documentation only
178+
179+
## [2026-06-14 11:00] - Spot-schedule dynamic event-driven provider watch (Phase 3)
180+
181+
**Author:** Erick Bourgeois
182+
183+
### Changed
184+
- `src/reconcilers/spot_schedule_watch.rs`: NEW. `ReverseIndex` (pure:
185+
`(gvk, ns, name)` → referencing `ScheduledMachine`s, with per-SM record so a
186+
changed/removed `spec.spotSchedule` updates precisely) + `provider_key_for`.
187+
`SpotScheduleWatchManager` owns the index and lazily runs **one
188+
`kube::runtime::watcher` stream per referenced provider GVK** (event-driven,
189+
not polling); `sync_watchers` reconciles the running watcher set against the
190+
index's referenced GVKs (spawn new, abort orphaned). Per-GVK task resolves the
191+
`ApiResource` via `discovery::pinned_kind` with back-off
192+
(`SPOT_SCHEDULE_DISCOVERY_RETRY_SECS`, handles CRD-installed-late and
193+
CRD-deleted-while-watched), maps each provider event back through the index,
194+
and enqueues the referencing SMs.
195+
- `src/main.rs`: a standalone `ScheduledMachine` reflector feeds SM apply/delete
196+
into the manager (index rebuilt from the reflector's initial list on restart —
197+
nothing stored outside Kubernetes); a second `Controller::reconcile_on`
198+
consumes the manager's mpsc so provider `status.active` flips trigger SM
199+
reconciles at watch latency.
200+
- `src/constants.rs`: `SPOT_SCHEDULE_EVENT_CHANNEL_CAP`,
201+
`SPOT_SCHEDULE_DISCOVERY_RETRY_SECS`.
202+
- `src/reconcilers/mod.rs`: export the watch types + `compose_should_be_active`.
203+
- Tests: `spot_schedule_watch_tests.rs` (13 — ReverseIndex register/replace/
204+
deregister/lookup/gvks, `provider_key_for`, manager observe/forget watcher
205+
lifecycle incl. shared-GVK and watcher-stop-on-last-deref);
206+
`tests/integration_spot_schedule.rs` (4 — create→active, flip→inactive,
207+
delete→unresolved+hold/fail-inactive across a mock kube API, and the watch
208+
index→reconcile mapping). 658 lib tests green.
209+
- Docs: contract page status → Phase 3.
210+
211+
### Why
212+
Phase 3 of the spot-schedule provider roadmap: remove the Phase-2 pull latency.
213+
Provider status transitions are now a reconcile **trigger** (a true watch over
214+
the `spotschedules.5spot.finos.org` group), not something re-checked only on the
215+
controller's own timer — satisfying the project's watch-not-poll rule. Realizes
216+
the `rel-controller-spot-schedule-watch` relationship + `flow-spot-schedule-activation`
217+
already modeled in CALM at Phase 0 (no ADR/CALM change).
218+
219+
### Impact
220+
- [ ] Breaking change
221+
- [x] Requires cluster rollout (new controller image; RBAC for
222+
`spotschedules.5spot.finos.org` get/list/watch lands in Phase 4)
223+
- [ ] Config change only
224+
- [ ] Documentation only
225+
226+
## [2026-06-13 18:45] - Spot-schedule resolver + composition + status surface (Phase 2)
227+
228+
**Author:** Erick Bourgeois
229+
230+
### Changed
231+
- `src/reconcilers/spot_schedule.rs`: NEW resolver module. `SpotScheduleVerdict`
232+
enum (`Active`/`Inactive`/`Unresolved`); pure `verdict_from_status` (duck-typed
233+
`status.active` + `Ready` extraction); async `resolve_spot_schedule`
234+
(`discovery::pinned_kind` + `get_opt`). Transient API errors propagate (retry);
235+
CRD-not-installed / object-absent / status-missing / not-Ready are `Unresolved`
236+
verdicts, never errors or panics. **Ready semantics finalized:** absent `Ready`
237+
`active` authoritative; present non-`True` `Ready` ⇒ unresolved (resolves an
238+
internal inconsistency in ADR 0006 §4 / contract table — both updated).
239+
- `src/reconcilers/helpers.rs`: NEW `compose_should_be_active` — AND composition
240+
of the inline schedule and the provider verdict, with hold-last-state on
241+
Unresolved (fail-inactive when never resolved). `killSwitch` precedence is
242+
upstream.
243+
- `src/crd.rs`: replaced Phase-1 `effective_schedule()` with `is_enabled()` (a
244+
spotSchedule-only machine is always enabled; the provider governs activity).
245+
NEW `SpotScheduleStatus` + `ScheduledMachineStatus.spot_schedule` field (the
246+
durable hold-last-state surface: `resolved`/`active`/`reason`/`message`/
247+
`providerGeneration`/`lastTransitionTime`).
248+
- `src/reconcilers/scheduled_machine.rs`: `reconcile_inner` now evaluates the
249+
optional inline schedule, resolves the provider (pull-on-reconcile), composes
250+
`should_be_active`, records metrics, and patches `status.spotSchedule`
251+
(`persist_spot_schedule_status`, a `/status` merge patch that coexists with the
252+
phase/condition writes). Handler `enabled` guards use `is_enabled()`.
253+
- `src/metrics.rs`: NEW `fivespot_spot_schedule_resolutions_total{namespace,kind,result}`,
254+
`_resolution_errors_total{namespace,kind,reason}`,
255+
`_transitions_total{namespace,kind}` (+ record fns; bounded cardinality, never
256+
keyed by name).
257+
- `deploy/crds/scheduledmachine.yaml` regenerated (status.spotSchedule);
258+
`docs/src/reference/api.md` regenerated.
259+
- Tests: `spot_schedule_tests.rs` (verdict truth table + 3 mock-client resolve
260+
paths), `helpers_tests.rs` (12-case composition truth table),
261+
`scheduled_machine_tests.rs` (persist `/status` patch), `metrics_tests.rs`,
262+
`crd_tests.rs` (status round-trip + is_enabled). 645 lib tests green.
263+
- Docs: ADR 0006 + contract page `Ready` clarification; contract page status →
264+
Phase 2; `docs/src/operations/monitoring.md` spot-schedule metrics table.
265+
266+
### Why
267+
Phase 2 of the spot-schedule provider roadmap: make a `spec.spotSchedule`
268+
reference actually drive the machine's active/inactive decision (pull-on-reconcile),
269+
with the ADR-specified hold-last-state fail-safe and an observable status surface.
270+
The event-driven watch (Phase 3) and the reference-provider controller (Phase 5)
271+
build on this.
272+
273+
### Impact
274+
- [ ] Breaking change
275+
- [x] Requires cluster rollout (apply regenerated CRDs + new controller image)
276+
- [ ] Config change only
277+
- [ ] Documentation only
278+
279+
>>>>>>> Stashed changes
12280
## [2026-06-13 16:30] - Spot-schedule provider API: CRD scaffolding + ScheduledMachine v1beta1 (Phase 1)
13281

14282
**Author:** Erick Bourgeois

.github/workflows/build.yaml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,38 @@ jobs:
159159
- name: Parse every .vex/*.json via vexctl merge
160160
run: make vex-validate
161161

162+
# ─────────────────────────────────────────────────────────────────────────────
163+
# Auto-VEX pre-submission gate — PR + push (ADR 0008).
164+
# The auto-VEX suppressions must be generated, reviewed, and signed off
165+
# (committed) BEFORE submission. This job regenerates .vex/auto/* from the
166+
# frozen .vex/snapshot/ with pinned @id/author/timestamp and byte-exact diffs
167+
# the result against the committed output. A mismatch hard-fails the build:
168+
# the contributor must run `make vex-auto`, review the suppressions, and commit.
169+
# This is the upstream human checkpoint; the Security team's release-time
170+
# counter-signature (see docs/src/security/vex.md) remains the downstream net.
171+
# ─────────────────────────────────────────────────────────────────────────────
172+
validate-autovex:
173+
name: 🔏 Validate Auto-VEX Sign-off
174+
if: github.event_name == 'pull_request' || github.event_name == 'push'
175+
runs-on: ubuntu-latest
176+
needs: [verify-commits]
177+
steps:
178+
- name: Checkout code
179+
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
180+
181+
- name: Install Rust toolchain
182+
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable (moving ref — re-pin quarterly)
183+
184+
- name: Cache cargo dependencies
185+
uses: firestoned/github-actions/rust/cache-cargo@53b483254bc648903c364ee3c73a546d0936a91e # v1.3.6
186+
187+
# Regenerate .vex/auto/* from the committed .vex/snapshot/ and fail if the
188+
# committed output is stale. Output is byte-deterministic (pinned
189+
# @id/author/timestamp + CVE-sorted generators), so a plain git diff is the
190+
# whole gate.
191+
- name: Verify committed auto-VEX matches a fresh regeneration
192+
run: make vex-auto-check
193+
162194
# ─────────────────────────────────────────────────────────────────────────────
163195
# Extract version — all events; downstream build jobs depend on this.
164196
# format/clippy are independent PR quality checks and do not gate the build.

.vex/auto/vex.auto-presence.json

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"@context": "https://openvex.dev/ns/v0.2.0",
3+
"@id": "https://github.com/finos/5-spot/.vex/auto/presence",
4+
"author": "auto-vex-presence",
5+
"timestamp": "2026-06-14T00:00:00Z",
6+
"version": 1,
7+
"statements": []
8+
}
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"@context": "https://openvex.dev/ns/v0.2.0",
3+
"@id": "https://github.com/finos/5-spot/.vex/auto/reachability",
4+
"author": "auto-vex-reachability",
5+
"timestamp": "2026-06-14T00:00:00Z",
6+
"version": 1,
7+
"statements": []
8+
}

0 commit comments

Comments
 (0)