Skip to content

Commit d8555eb

Browse files
westgatewestgate
authored andcommitted
S203: composition elevation sprint + deep debt execution
Standardize compute.dispatch wire contract — all 8 dispatch handlers share canonical {domain, operation, job_id, status, output, error, metadata} envelope for primalSpring typed extractors. Smart refactor 4 large files (test extraction). Deprecate get_primal_default_port, evolve GPU buffer unsafe to NonNull pattern, centralize discovery ports, resolve clippy suppressions, clean deny.toml advisories. Document wire contract in specs/DISPATCH_WIRE_CONTRACT.md. Update all root docs to S203. Made-with: Cursor
1 parent 6e289c9 commit d8555eb

54 files changed

Lines changed: 3059 additions & 2360 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -196,6 +196,32 @@ jobs:
196196
fi
197197
echo "No secrets found in working tree."
198198
199+
fuzz:
200+
name: Fuzz (libFuzzer smoke)
201+
runs-on: ubuntu-latest
202+
strategy:
203+
fail-fast: false
204+
matrix:
205+
target:
206+
- fuzz_jsonrpc_parse
207+
- fuzz_config_toml
208+
- fuzz_btsp_framing
209+
steps:
210+
- uses: actions/checkout@v4
211+
- uses: dtolnay/rust-toolchain@nightly
212+
- uses: Swatinem/rust-cache@v2
213+
with:
214+
workspaces: fuzz
215+
- name: Install system dependencies
216+
run: |
217+
sudo apt-get update
218+
sudo apt-get install -y libssl-dev pkg-config build-essential
219+
- name: Install cargo-fuzz
220+
uses: taiki-e/install-action@cargo-fuzz
221+
- name: Run fuzz target (${{ matrix.target }})
222+
working-directory: fuzz
223+
run: cargo fuzz run ${{ matrix.target }} -- -max_total_time=120 -rss_limit_mb=4096
224+
199225
security-audit:
200226
name: Security Audit
201227
runs-on: ubuntu-latest

CHANGELOG.md

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,51 @@ All notable changes to ToadStool will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8-
## [Unreleased] - April 11, 2026 (Sessions 43-202)
8+
## [Unreleased] - April 12, 2026 (Sessions 43-203)
9+
10+
### Session S203 (Apr 12, 2026) — Composition Elevation Sprint + Deep Debt Execution
11+
12+
#### Dispatch Wire Contract Standardization (Blocking Composition)
13+
- STANDARDIZED: All 8 `compute.dispatch.*` handlers share canonical envelope: `{domain, operation, job_id, status, output, error, metadata}`
14+
- EVOLVED: `shader.dispatch` domain `"shader.dispatch"``"compute.dispatch"` with `operation: "shader"`
15+
- EVOLVED: Pipeline domain `"compute.dispatch.pipeline"``"compute.dispatch"` with `operation: "pipeline.submit"` / `"pipeline.status"`
16+
- EVOLVED: Status field from compound strings (`"failed: msg"`) to clean enum values + separate `error` field
17+
- EVOLVED: Inline result/bdf/workgroup fields → structured `output` + `metadata` objects
18+
- ADDED: `DispatchStatus::as_str()` and `PipelineStatus::as_str()` for wire-stable status tags
19+
- ADDED: `specs/DISPATCH_WIRE_CONTRACT.md` — full wire contract documentation for primalSpring typed extractors
20+
21+
#### Smart File Refactoring (4 production files >550 LOC)
22+
- EXTRACTED: `server/background/mod.rs` tests → `tests.rs` (608→72 lines)
23+
- EXTRACTED: `distributed/federation/mod.rs` tests → `tests.rs` (594→109 lines)
24+
- EXTRACTED: `encryption/provider.rs` tests → `provider_tests.rs` (568→257 lines)
25+
- EXTRACTED: `runtime/universal/runtime.rs` tests → `runtime_tests.rs`, `RuntimeStats``stats.rs` (576→249 lines)
26+
27+
#### Primal Name Evolution
28+
- DEPRECATED: `get_primal_default_port` with migration path to `resolve_capability_port`
29+
- MIGRATED: All callers to capability identifiers (COORDINATION, SECURITY, STORAGE, PLATFORM)
30+
31+
#### Unsafe Code Evolution
32+
- EVOLVED: GPU buffer `access.rs``from_raw_parts``NonNull::slice_from_raw_parts` (safe metadata) + scoped `unsafe { .as_ref() }`
33+
- DOCUMENTED: Safety contracts narrowed to aliasing-only invariant
34+
35+
#### Port Centralization
36+
- CENTRALIZED: Discovery fallback ports from 4 scattered modules → `common/constants/discovery_ports.rs`
37+
- ADDED: Re-exports via `config/defaults/ports.rs` as single registry
38+
39+
#### Clippy Suppression Cleanup
40+
- RESOLVED: `unused_self` in `estimator.rs` — converted helpers to associated functions
41+
- RESOLVED: `cast_sign_loss`/`cast_possible_wrap` in `auth/mod.rs` — eliminated `as` casts
42+
- DOCUMENTED: `needless_pass_by_ref_mut` in buffer `access.rs` as soundness requirement
43+
44+
#### deny.toml Advisory Cleanup
45+
- REMOVED: 6 stale RUSTSEC ignores (no longer in dependency graph)
46+
- UPDATED: RUSTSEC-2024-0436 reason (paste via statrs→nalgebra→simba chain)
47+
48+
#### Quality Gates
49+
- `cargo fmt`: PASS
50+
- `cargo clippy --workspace --all-targets`: PASS (0 warnings)
51+
- `cargo doc --workspace --no-deps`: PASS (0 warnings)
52+
- `cargo test --workspace`: PASS (0 failures)
953

1054
### Session S202 (Apr 11, 2026) — Deep Debt Execution: Capability-Based Evolution
1155

DEBT.md

Lines changed: 115 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Active Technical Debt Register
22

3-
**Date**: April 11, 2026 — S202
3+
**Date**: April 12, 2026 — S203
44
**Philosophy**: Math is universal, precision is silicon. Workarounds are
55
short-term solutions that increase debt. We aim to solve deep debt over
66
iterations, evolving toward vendor-agnostic, capability-based solutions.
@@ -43,26 +43,16 @@ Dead u8 alignment check removed S197. Evolution: fuzz the access paths, consider
4343
`NonNull::slice_from_raw_parts` for fat-pointer representation.
4444
Files: `buffer/access.rs`.
4545

46-
### D-FUZZ-TARGETS — CI & corpus (infra landed S197, remaining work)
46+
### D-FUZZ-TARGETS — seed corpus & extended campaigns
4747
**Scope**: Workspace | **Dir**: `fuzz/`
48-
Initial `cargo-fuzz` / `libfuzzer` infrastructure added (S197). Three targets:
49-
`fuzz_jsonrpc_parse` (JSON-RPC 2.0 deser), `fuzz_config_toml` (config deser +
50-
validation), `fuzz_btsp_framing` (BTSP length-prefixed frame decode).
51-
Remaining: integrate into CI, add seed corpus, run extended campaigns, add
52-
proptest bridge for property-based input generation.
48+
Three fuzz targets landed (S197) and **CI smoke integration landed (S203)**.
49+
`ci.yml` now runs all three targets with `cargo fuzz run` (2min/target, nightly).
50+
Remaining: add seed corpus from real JSON-RPC traffic, run extended campaigns,
51+
add proptest bridge for property-based input generation.
5352
See also: `D-FUZZ-TARGETS-UNSAFE` (GPU buffer access paths).
54-
Files: `fuzz/Cargo.toml`, `fuzz/fuzz_targets/*.rs`.
53+
Files: `fuzz/Cargo.toml`, `fuzz/fuzz_targets/*.rs`, `.github/workflows/ci.yml`.
5554

5655

57-
### D-RUSTIX-DISPLAY-038
58-
**Crate**: `runtime/display` | **Dep**: `rustix 0.38`
59-
V4L2 ioctl wrappers use `Getter`/`Updater`/`Setter` convenience types removed in
60-
rustix 1.x. Migration requires rewriting ~3 unsafe ioctl helpers to the new `Ioctl`
61-
trait pattern (as hw-safe and nvpmu already use). Low urgency: display crate is a
62-
leaf and the 0.38 version is safe; the only impact is having two rustix majors in
63-
the dependency tree.
64-
Files: `v4l2/ioctl.rs`, `drm/device.rs`, `v4l2/device.rs`.
65-
6656
## Known Limitations (not actionable debt)
6757

6858
### D-ASYNC-DYN-MARKERS — Rust language constraint
@@ -72,6 +62,114 @@ stable in Rust. Cannot resolve until Rust stabilizes this feature. The `#[async_
7262
dependency is pure Rust (proc-macro) and zero-overhead at runtime for non-dyn paths.
7363
**Not actionable** — resolves when Rust stabilizes the feature. Markers are accurate documentation.
7464

65+
## S203 Resolved Debt (Deep Audit & Evolution Execution)
66+
67+
### D-RUSTIX-DISPLAY-038 — RESOLVED S203
68+
**Crate**: `runtime/display` | **Dep**: `rustix 0.38``1.1`
69+
V4L2 ioctl wrappers migrated from `ReadOpcode`/`WriteOpcode`/`ReadWriteOpcode` +
70+
`Getter`/`Updater`/`Setter` to rustix 1.x `ioctl::opcode::{read,write,read_write}`
71+
const functions via type-concrete macros (`v4l2_getter!`, `v4l2_updater!`, `v4l2_setter!`).
72+
Eliminates duplicate `rustix` majors from the dependency tree. Unused features
73+
(`mm`, `process`, `io_uring`) dropped; only `fs` + `all-apis` remain.
74+
Files: `Cargo.toml`, `v4l2/ioctl.rs`.
75+
76+
### D-CLIPPY-WARNINGS — RESOLVED S203
77+
Four clippy warnings eliminated by evolving dead code into production use:
78+
- `DispatchStatus::Running` wired in `submit.rs` (set before dispatch)
79+
- `PipelineStageRequest.substrate` wired through to `PipelineStageResult` (visible in responses)
80+
- `PipelineStatus::Failed` wired for graph validation failures (tracked pipelines)
81+
- Redundant closure in `wire_l3.rs` replaced with method reference
82+
83+
### D-DOC-EMPTY-CODEBLOCK — RESOLVED S203
84+
Empty Rust code block in `cli/src/ecosystem/services/mod.rs` changed from
85+
`rust,ignore` to `text` (commented-out legacy code is not valid Rust).
86+
87+
### D-NVPMU-STALE-ALLOWS — RESOLVED S203
88+
Stale `#[allow(unsafe_code)]` removed from `bar0` and `init` modules in `nvpmu`
89+
(neither contains `unsafe` blocks — they use safe `hw-safe` wrappers).
90+
91+
### D-MMIO-ALIGNMENT — EVOLVED S203
92+
`volatile_mmio.rs` alignment checks evolved from `debug_assert!` to release-mode
93+
`MmioError::Misaligned` error returns. Prevents potential UB from misaligned
94+
volatile reads/writes in release builds.
95+
96+
### D-FUZZ-CI — RESOLVED S203
97+
Fuzz smoke job added to `.github/workflows/ci.yml`: three targets
98+
(`fuzz_jsonrpc_parse`, `fuzz_config_toml`, `fuzz_btsp_framing`) run with
99+
`-max_total_time=120` on nightly via `cargo-fuzz`. Matrix strategy with
100+
`fail-fast: false`.
101+
102+
### D-BYOB-HARDCODED — RESOLVED S203
103+
BYOB config hardcoded ports and timeouts extracted to named constants
104+
(`DEFAULT_MAX_CONCURRENT_DEPLOYMENTS`, `COMMON_WEB_SERVICE_PORTS`, etc.).
105+
Coordinator port now dynamically appended rather than statically positioned.
106+
107+
### D-DISPATCH-RESPONSE-SHAPE — RESOLVED S203
108+
**Scope**: `server/dispatch/` (all handlers) | **Blocking**: Composition Elevation
109+
primalSpring's typed extractors (`extract_rpc_result<T>` / `extract_rpc_dispatch<T>`)
110+
required a consistent envelope across all dispatch variants. Previously:
111+
- `shader.dispatch` responses used `"domain": "shader.dispatch"` and omitted `"operation"`
112+
- Pipeline responses used `"domain": "compute.dispatch.pipeline"` with flat `stage_results`
113+
- Status fields embedded error details in compound strings (`"failed: msg"`)
114+
- `result` was sometimes present, sometimes absent, sometimes null
115+
116+
All 8 dispatch operations now share a single canonical envelope:
117+
`{ domain, operation, job_id, status, output, error, metadata }`.
118+
Status field is always a clean enum value (`submitted|running|completed|failed|partial_failure`).
119+
Error details moved to dedicated `error` field. Type-specific context in `metadata`.
120+
Wire contract documented in `specs/DISPATCH_WIRE_CONTRACT.md`.
121+
Completes Node Atomic chain: coralReef → toadStool → barraCuda composition parity.
122+
123+
### D-DISPATCH-STATUS-COMPOUND — RESOLVED S203
124+
**Crate**: `server/dispatch/types.rs`
125+
`DispatchStatus::Display` and `PipelineStatus::Display` produced compound strings
126+
(`"failed: msg"`, `"running:stage_id"`) that leaked internal state into the wire
127+
`status` field. Added `as_str()` methods returning clean wire-stable enum tags.
128+
`Display` impl preserved for debug/logging use.
129+
130+
### D-LARGE-FILE-REFACTOR — RESOLVED S203
131+
**Scope**: Workspace (6 production files >550 LOC)
132+
Smart test extraction from oversized `mod.rs` / `lib.rs` files:
133+
- `server/src/background/mod.rs` 608→72 lines (tests → `tests.rs`)
134+
- `distributed/src/cloud/federation/mod.rs` 594→109 (tests → `tests.rs`)
135+
- `core/toadstool/src/encryption/provider.rs` 568→257 (tests → `provider_tests.rs`)
136+
- `runtime/universal/src/runtime.rs` 576→249 (tests → `runtime_tests.rs`, `RuntimeStats``stats.rs`)
137+
138+
### D-PRIMAL-PORT-DEPRECATION — RESOLVED S203
139+
**Crate**: `core/config/src/config_utils/network.rs`
140+
`get_primal_default_port` (maps legacy primal names to capability ports) deprecated
141+
with migration path. All callers migrated to `resolve_capability_port` directly
142+
with capability identifiers (`COORDINATION`, `SECURITY`, `STORAGE`, `PLATFORM`).
143+
144+
### D-DISCOVERY-PORT-CENTRALIZATION — RESOLVED S203
145+
**Scope**: `core/common`, `core/config`, `runtime/display`
146+
Scattered fallback port definitions (`DISCOVERY_HTTP_PORT_FALLBACK` 8080,
147+
`TOADSTOOL_DISCOVERY_FALLBACK_PORT` 9080, `DISPLAY_IPC_FALLBACK_PORT` 8091)
148+
centralized into `common/constants/discovery_ports.rs` with re-exports via
149+
`config/defaults/ports.rs`.
150+
151+
### D-CLIPPY-SUPPRESSIONS — RESOLVED S203
152+
Resolved rather than suppressed:
153+
- `server/resource_estimator/estimator.rs`: `unused_self` → converted helpers to
154+
associated functions
155+
- `core/toadstool/biomeos_integration/auth/mod.rs`: `cast_sign_loss`/`cast_possible_wrap`
156+
→ eliminated `as` casts with direct `u64` from `Duration::as_secs()`
157+
- `runtime/gpu/buffer/access.rs`: `needless_pass_by_ref_mut` → documented as
158+
soundness requirement (exclusive borrow prevents aliased mutable GPU access)
159+
160+
### D-UNSAFE-BUFFER-EVOLUTION — RESOLVED S203
161+
**Crate**: `runtime/gpu/src/unified_memory/buffer/access.rs`
162+
`from_raw_parts`/`from_raw_parts_mut` evolved to `NonNull::slice_from_raw_parts`
163+
(safe metadata construction) + `unsafe { .as_ref()/.as_mut() }` (aliasing contract
164+
only). Safety documentation updated to match the narrower invariant.
165+
166+
### D-DENY-TOML-STALE-ADVISORIES — RESOLVED S203
167+
**Scope**: `deny.toml`
168+
Six stale RUSTSEC ignores removed (advisories no longer in dependency graph):
169+
RUSTSEC-2024-0387, RUSTSEC-2024-0438, RUSTSEC-2025-0046, RUSTSEC-2025-0118,
170+
RUSTSEC-2026-0020, RUSTSEC-2026-0021. Only RUSTSEC-2024-0436 (paste via
171+
statrs→nalgebra→simba chain, INFO-level unmaintained) remains with updated reason.
172+
75173
## S202 Resolved Debt (Deep Debt Execution: Capability-Based Evolution)
76174

77175
### D-HARDCODED-PRIMAL-LITERALS — RESOLVED S202

DOCUMENTATION.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ToadStool Documentation Hub
22

3-
**Last Updated**: April 11, 2026 — S202
3+
**Last Updated**: April 12, 2026 — S203
44

55
---
66

NEXT_STEPS.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ToadStool -- Next Steps
22

3-
**Updated**: April 11, 2026 -- S202 (Deep Debt Execution: Capability-Based Evolution)
3+
**Updated**: April 12, 2026 — S203 (Deep Debt Execution: Capability-Based Evolution)
44
**Status**: Production-grade | Rust edition **2024** (MSRV 1.85) | **AGPL-3.0-or-later** | **All quality gates green** | 21,600+ tests (0 failures) | **~69 JSON-RPC methods** | Wire Standard L3 (partial) | Zero C FFI deps (ecoBin v3.0) | Zero production unwraps | IPC-first | **43/43 crates with `unsafe_code` lint policy** | **34 unsafe blocks** (all in hw containment) | **~80 justified #[allow]** | **0 production TODOs** | **~3m30s test runtime** | **rustix 1.x everywhere except display** | **capability-based primal references (no hardcoded names)**
55
**Latest**: S202 — Deep debt execution: hardcoded `"toadstool"` literals → `PRIMAL_NAME`, `"coral_reef_available"``"shader_compiler_available"`, ~15 doc comments evolved to capability wording, `serialport` feature-gated, dead code removed, `jsonrpc_server.rs` DRY'd. S201 (prior): primalSpring gap closure, +46 tests
66

0 commit comments

Comments
 (0)