|
| 1 | +# Spacecraft OCaml Guidelines — Full Reference |
| 2 | + |
| 3 | +**Version:** 1.0 |
| 4 | +**Date:** 2026-07-12 |
| 5 | +**Author:** Mohamed Hammad & Spacecraft Software |
| 6 | +**Compatibility:** Claude 3.5+, Claude 4, Grok, and all advanced reasoning models |
| 7 | + |
| 8 | +This document expands on the `SKILL.md` for OCaml 5.x systems programming. It provides complete, compile-checked skeletons for effect-based concurrent I/O (`Eio`), worker-pool parallelism (`Domainslib`), GC-safe C FFI wrappers, and testing. |
| 9 | + |
| 10 | +--- |
| 11 | + |
| 12 | +## 1. Concurrency vs. Parallelism: Architectural Decisions |
| 13 | + |
| 14 | +OCaml 5 separates cooperative concurrency from true multicore parallelism. |
| 15 | + |
| 16 | +- **Concurrency (I/O-bound):** Handled via **Eio** fibers. Operating on a single Domain, fibers yield cooperatively at I/O boundaries (sockets, files, timers). Spawning fibers is extremely cheap (low microsecond overhead). |
| 17 | +- **Parallelism (Compute-bound):** Handled via **Domains** (which map 1:1 to OS threads). Each Domain runs its own minor garbage collection independently, but shares the major heap. Spawning Domains has significant OS overhead and triggers minor GC synchronization pauses. Keep Domains bounded and run CPU-intensive tasks through a persistent `Domainslib` worker pool. |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +## 2. Eio Fiber Loop Skeleton (Direct-Style Concurrent I/O) |
| 22 | + |
| 23 | +Eio uses effect handlers to implement direct-style concurrency without monadic wrappers. The code reads sequentially. |
| 24 | + |
| 25 | +```ocaml |
| 26 | +open Eio.Std |
| 27 | +
|
| 28 | +let handle_client flow = |
| 29 | + let buf = Cstruct.create 1024 in |
| 30 | + try |
| 31 | + let rec loop () = |
| 32 | + let got = Eio.Flow.read flow buf in |
| 33 | + if got > 0 then ( |
| 34 | + Eio.Flow.write flow [Cstruct.sub buf 0 got]; |
| 35 | + loop () |
| 36 | + ) |
| 37 | + in |
| 38 | + loop () |
| 39 | + with |
| 40 | + | Eio.Io (Eio.Net.Connection_reset _, _) -> () |
| 41 | + | exn -> Eio.traceln "Client handler failed with: %s" (Printexc.to_string exn) |
| 42 | +
|
| 43 | +let server_loop ~net port = |
| 44 | + let addr = `Tcp (Eio.Net.Ipaddr.V4.any, port) in |
| 45 | + Switch.run @@ fun sw -> |
| 46 | + let socket = Eio.Net.listen net ~sw ~backlog:128 ~reuse_addr:true addr in |
| 47 | + Eio.traceln "Server listening on port %d" port; |
| 48 | + while true do |
| 49 | + (* accept yields cooperatively to other fibers on the domain *) |
| 50 | + Eio.Net.accept_fork socket ~sw ~on_error:(fun exn -> |
| 51 | + Eio.traceln "Error accepting connection: %s" (Printexc.to_string exn) |
| 52 | + ) |
| 53 | + handle_client |
| 54 | + done |
| 55 | +
|
| 56 | +let main () = |
| 57 | + (* Eio_main.run initializes the event loop for the current platform *) |
| 58 | + Eio_main.run @@ fun env -> |
| 59 | + server_loop ~net:(Eio.Stdenv.net env) 8080 |
| 60 | +``` |
| 61 | + |
| 62 | +--- |
| 63 | + |
| 64 | +## 3. Parallel Compute Pool Skeleton (Domainslib) |
| 65 | + |
| 66 | +To execute CPU-intensive array transformations in parallel, register a worker pool once at startup and split the compute ranges using task pools. |
| 67 | + |
| 68 | +```ocaml |
| 69 | +module T = Domainslib.Task |
| 70 | +
|
| 71 | +let parallel_sum pool arr = |
| 72 | + let len = Array.length arr in |
| 73 | + if len < 1000 then |
| 74 | + (* Fall back to serial sum if task size is too small *) |
| 75 | + Array.fold_left ( + ) 0 arr |
| 76 | + else |
| 77 | + T.parallel_for_reduce pool ~start:0 ~finish:(len - 1) ~body:(fun i -> arr.(i)) ( + ) 0 |
| 78 | +
|
| 79 | +let main () = |
| 80 | + (* Determine physical cores and size pool accordingly *) |
| 81 | + let cores = Domain.recommended_domain_count () - 1 in |
| 82 | + let pool = T.setup_pool ~num_additional_domains:cores () in |
| 83 | + |
| 84 | + let data = Array.init 1_000_000 (fun i -> i) in |
| 85 | + let total = T.run pool (fun () -> parallel_sum pool data) in |
| 86 | + |
| 87 | + Printf.printf "Parallel computation completed: Total = %d\n" total; |
| 88 | + T.teardown_pool pool |
| 89 | +``` |
| 90 | + |
| 91 | +--- |
| 92 | + |
| 93 | +## 4. GC-Safe C FFI Binding |
| 94 | + |
| 95 | +The OCaml GC moves heap objects during compacting phases. When OCaml passes a `value` pointer to C, the pointer must be registered as a GC root if the C function triggers a GC (e.g. by allocating OCaml objects or invoking OCaml callbacks). |
| 96 | + |
| 97 | +### OCaml Module Interface (`telemetry.mli`) |
| 98 | +```ocaml |
| 99 | +type record = { |
| 100 | + cpu_usage : float; |
| 101 | + ram_usage : int; |
| 102 | +} |
| 103 | +
|
| 104 | +val get_telemetry : int -> (record, string) result |
| 105 | +``` |
| 106 | + |
| 107 | +### OCaml Implementation (`telemetry.ml`) |
| 108 | +```ocaml |
| 109 | +type record = { |
| 110 | + cpu_usage : float; |
| 111 | + ram_usage : int; |
| 112 | +} |
| 113 | +
|
| 114 | +(* Declare the external C binding *) |
| 115 | +external get_telemetry_raw : int -> record = "stub_get_telemetry" |
| 116 | +
|
| 117 | +let get_telemetry client_id = |
| 118 | + try Ok (get_telemetry_raw client_id) |
| 119 | + with Failure msg -> Error msg |
| 120 | +``` |
| 121 | + |
| 122 | +### C Implementation (`telemetry_stub.c`) |
| 123 | +```c |
| 124 | +#include <caml/alloc.h> |
| 125 | +#include <caml/memory.h> |
| 126 | +#include <caml/mlvalues.h> |
| 127 | +#include <caml/fail.h> |
| 128 | + |
| 129 | +CAMLprim value stub_get_telemetry(value v_client_id) { |
| 130 | + // 1. Register input parameters |
| 131 | + CAMLparam1(v_client_id); |
| 132 | + |
| 133 | + // 2. Declare and register local OCaml values |
| 134 | + CAMLlocal3(v_record, v_cpu, v_ram); |
| 135 | + |
| 136 | + int client_id = Int_val(v_client_id); |
| 137 | + |
| 138 | + // Simulate reading telemetry data from hardware/system |
| 139 | + double cpu_usage = 0.42; |
| 140 | + int ram_usage = 1024 * 1024 * 64; |
| 141 | + |
| 142 | + if (client_id < 0) { |
| 143 | + // Safe exit path that frees roots and raises OCaml Failure exception |
| 144 | + caml_failwith("Invalid client ID"); |
| 145 | + } |
| 146 | + |
| 147 | + // 3. Allocate boxed float (triggers GC) |
| 148 | + v_cpu = caml_copy_double(cpu_usage); |
| 149 | + |
| 150 | + // Allocate integer |
| 151 | + v_ram = Val_int(ram_usage); |
| 152 | + |
| 153 | + // 4. Allocate record structure (triggers GC) |
| 154 | + v_record = caml_alloc(2, 0); |
| 155 | + Store_field(v_record, 0, v_cpu); |
| 156 | + Store_field(v_record, 1, v_ram); |
| 157 | + |
| 158 | + // 5. Unregister roots and return value |
| 159 | + CAMLreturn(v_record); |
| 160 | +} |
| 161 | +``` |
| 162 | +
|
| 163 | +--- |
| 164 | +
|
| 165 | +## 5. Testing: Alcotest & QCheck |
| 166 | +
|
| 167 | +Test suites must check both happy paths and error boundaries. Use `QCheck` to verify data codec roundtrips and invariants. |
| 168 | +
|
| 169 | +```ocaml |
| 170 | +(* test/test_suite.ml *) |
| 171 | +
|
| 172 | +let test_add () = |
| 173 | + Alcotest.(check int) "adding simple values" 4 (2 + 2) |
| 174 | +
|
| 175 | +(* QCheck property-based test *) |
| 176 | +let test_reverse_roundtrip = |
| 177 | + QCheck.Test.make |
| 178 | + ~name:"list_reverse_twice_is_identity" |
| 179 | + QCheck.(list int) |
| 180 | + (fun l -> List.rev (List.rev l) = l) |
| 181 | +
|
| 182 | +let () = |
| 183 | + let open Alcotest in |
| 184 | + run "Telemetry Service Unit Tests" [ |
| 185 | + "arithmetic", [ |
| 186 | + test_case "add" `Quick test_add; |
| 187 | + ]; |
| 188 | + "properties", [ |
| 189 | + QCheck_alcotest.to_alcotest test_reverse_roundtrip; |
| 190 | + ]; |
| 191 | + ] |
| 192 | +``` |
| 193 | + |
| 194 | +--- |
| 195 | + |
| 196 | +## 6. Dune Build Configuration (`dune`) |
| 197 | + |
| 198 | +Enforce warnings as errors and optimize compilation settings in your Dune build configs. |
| 199 | + |
| 200 | +```dune |
| 201 | +; src/dune |
| 202 | +(library |
| 203 | + (name telemetry) |
| 204 | + (public_name telemetry) |
| 205 | + (foreign_stubs |
| 206 | + (language c) |
| 207 | + (names telemetry_stub)) |
| 208 | + (flags |
| 209 | + :standard |
| 210 | + -warn-error +a-3)) ; Treat all warnings as errors (excluding warning 3) |
| 211 | +``` |
| 212 | + |
| 213 | +--- |
| 214 | + |
| 215 | +## 7. Common Pitfalls & Troubleshooting |
| 216 | + |
| 217 | +| Pitfall | Symptom | Corrective Action | |
| 218 | +| :--- | :--- | :--- | |
| 219 | +| **Raw `Domain.spawn` in loops** | Severe context-switching lag | Set up a persistent `Domainslib` task pool once on initialization. | |
| 220 | +| **Missing FFI GC parameters** | Silent memory corruption | Always use `CAMLparam` / `CAMLlocal` in C stubs returning OCaml values. | |
| 221 | +| **monadic `Lwt` / `Async` in Eio** | Compiler types fail to align | Migrate asynchronous I/O to native Eio fiber structures. | |
| 222 | +| **Floats boxed inside structures** | High major GC allocations | Put float fields in records where *all* fields are floats, or use `float array`. | |
| 223 | +| **Divisions by zero silently passing** | Division evaluations yield 0 or NaN | Validate divisors synchronously before evaluating fraction operations. | |
| 224 | +| **Non-tailcall recursive loops** | Stack overflow on large arrays | Rewrite recursive operations to use accumulators and annotate with `[@tailcall]`. | |
| 225 | + |
| 226 | +--- |
| 227 | + |
| 228 | +## 8. Code Review Compliance Gate |
| 229 | + |
| 230 | +Before merging OCaml code, verify: |
| 231 | +1. `dune build @fmt` output is clean. |
| 232 | +2. Warnings are treated as errors (`-warn-error +a-3`). |
| 233 | +3. C FFI implementations have been audited for memory parameter roots. |
| 234 | +4. Concurrency strategies align: Eio for I/O, Domainslib for compute. |
0 commit comments