Skip to content

PROGRESS.md

Latest commit

 

History

History
281 lines (238 loc) · 13.7 KB

PROGRESS.md

File metadata and controls

281 lines (238 loc) · 13.7 KB

Mantis SDK — Project Progress

This document tracks the global advancement of the Mantis SDK. Agents must update this file when completing meaningful work. See philosophy/fin_sdk_oss_blueprint.md for full roadmap details.

Phase 0 — Project Bootstrap

Status: Complete | Completed: 2026-03-25

  • Rust workspace with crate skeletons (mantis-core, mantis-types, mantis-queue, mantis-bench, mantis-layout, mantis-verify)
  • CI pipeline: fmt, clippy, nextest, no_std test, doc, deny, miri, careful, coverage, codecov
  • Nightly CI: mutants, extended miri, full coverage, ASM toggle, ASM inspection, kani proofs, fuzz
  • Verification CI: kani proofs (4), bolero property tests (4), differential tests (3)
  • Benchmark regression CI with 5% threshold, artifact upload, PR annotations
  • Dependabot for cargo + actions
  • Coding guidelines (CLAUDE.md)
  • Unsafe policy (docs/UNSAFE.md)
  • Justfile task runner
  • Toolchain configs: rust-toolchain.toml, clippy.toml, deny.toml
  • Strategy pattern traits: IndexStrategy, PushPolicy, Instrumentation
  • Core newtypes: SeqNum, SlotIndex, QueueError
  • Benchmark counter trait + InstantCounter fallback
  • Layout inspector CLI
  • Constantine reference patterns documented

Not in scope (deferred):

  • API stability policy draft
  • Fuzz harness skeleton (placeholder only)
  • Topology visualization

Phase 1 — Minimal Useful Core

Status: In Progress | Started: 2026-03-25

1.1 SPSC Ring Buffer (mantis-queue)

Status: Complete

  • Core ring buffer engine with strategy pattern
  • raw submodule with unsafe slot operations
  • Power-of-2 masked index implementation
  • Cache-padded variant to prevent false sharing
  • Portable baseline implementation
  • Platform-specific atomics (x86_64 / ARM64)
  • Preset type aliases (SpscRing, SpscRingHeap, SpscRingInstrumented)
  • Unit tests (23 unit + 7 integration + 1 stress)
  • Miri validation (31/31 tests pass, zero UB)
  • Kani bounded model checking proofs (4 proofs)
  • Bolero property-based tests (4 properties)
  • Differential testing across strategy variants (3 comparisons)

1.2 Benchmark Harness (mantis-bench)

Status: Complete

  • RDTSC + lfence cycle counter (x86_64)
  • kperf / mach_absolute_time counter (macOS ARM64)
  • clock_gettime counter (Linux ARM64)
  • Criterion integration with JSON export
  • MantisMeasurement criterion Measurement trait
  • BenchReport with CPU name, compiler, full metrics schema
  • External contender benchmarks (bench-contenders feature)
    • rtrb
    • crossbeam
  • Benchmark workload shapes: single-item, burst, full-drain
  • Godbolt ASM inspection script

1.8 Platform Abstractions (mantis-platform)

Status: Complete

  • Constant-time types (Ct, CTBool, Carry, Borrow)
  • Constant-time arithmetic (ct_routines: eq, ne, lt, le, cneg, is_zero, is_msb_set)
  • Multiplexers with x86_64 cmov assembly (mux, ccopy, secret_lookup)
  • BearSSL constant-time division (div2n1n)
  • Carry/borrow arithmetic (addC, subB via widening)
  • Extended precision (WideMul, mul_acc, mul_double_acc)
  • Compiler hints (prefetch, prefetch_large)
  • Copy policies (CopyPolicy trait, DefaultCopyPolicy, SimdCopyPolicy)
  • Bit manipulation utilities (bithacks)
  • Platform configuration (config)
  • ISA assembler types (x86_64 + ARM64)
  • CPUID feature detection with OnceLock caching (x86_64)
  • RDTSC cycle counter (x86_64, moved from bench)
  • KperfCounter + PmuCounter (ARM64, moved from bench)
  • CycleCounter trait + Measurement + DefaultCounter
  • CachePadded (128-byte alignment)
  • CPU name detection
  • Migration: mantis-core, mantis-queue, mantis-bench updated to use platform
  • 162 tests, Miri validation, clippy clean

1.9 Fixed-Point Numeric Types (mantis-fixed, mantis-types expansion)

Status: Complete | Completed: 2026-04-03

  • FixedI64<const D: u8> compile-time-scaled fixed-point decimal backed by i64
  • Checked/saturating/wrapping add, sub, neg, abs
  • Explicit-rounding mul/div: checked_mul_trunc, checked_mul_round, checked_div_trunc, checked_div_round
  • Scalar integer mul/div: checked_mul_int, checked_div_int
  • Scale conversion: rescale_trunc, rescale_round, checked_rescale_exact
  • Display (D decimal places), Debug (raw + value), from_str_decimal parser
  • D <= 18 compile-time bound, validated scales: 2, 4, 6, 8
  • POW10_I64 const table in mantis-platform
  • Performance: hand-rolled decomposed division eliminates __divti3 runtime call
  • Contender benchmarks: faster than fixed crate (1.10ns vs 1.20ns mul) and rust_decimal (4x faster add)
  • Domain types in mantis-types: Side, Timestamp, OrderId
  • Hot-path types: Ticks(i64), Lots(i64) — pure integer, no decimal semantics
  • Semantic wrappers: UsdcAmount(FixedI64<6>), Probability(FixedI64<6>), BtcQty(FixedI64<8>)
  • InstrumentMeta<D> — tick/lot size conversion layer
  • 110 unit tests (mantis-fixed), 65 tests (mantis-types), 7 bolero property tests
  • Miri: 110/110 pass, zero UB
  • 2 fuzz targets (parse, display roundtrip)
  • Criterion benchmarks with contender comparison (rust_decimal, fixed crate)

1.3 Canonical Event Model (mantis-events)

Status: Complete | Completed: 2026-04-04

  • HotEvent — 64-byte, Copy, repr(C) envelope with header at offset 0
  • EventHeader — 24 bytes: recv_ts, seq, instrument_id, source_id, flags
  • EventBody — repr(C, u16) discriminated enum with 8 variants
  • EventKind — standalone u16 discriminant with 1:1 exhaustive mapping
  • EventFlags — u16 bitflags (IS_SNAPSHOT, LAST_IN_BATCH)
  • Market payloads: BookDeltaPayload (24B), TradePayload (24B), TopOfBookPayload (32B)
  • Execution payloads: OrderAckPayload (24B), FillPayload (32B), OrderRejectPayload (24B)
  • Control payloads: TimerPayload (8B), HeartbeatPayload (4B)
  • Supporting enums: UpdateAction, OrderStatus, RejectReason, TimerKind (all repr(u8))
  • Constructor helpers on HotEvent (const fn, #[must_use])
  • Const size assertions + authoritative layout tests in mantis-layout
  • Dependency firewall: depends on mantis-types only, NOT mantis-fixed
  • Prerequisites: InstrumentId(u32), SourceId(u16) in mantis-types, SeqNum hygiene fix
  • 57 tests, Miri validation (57/57 pass, zero UB), no_std clean

1.10 Sequence Lock (mantis-seqlock)

Status: Complete

  • Core SeqLock<T, C> with CopyPolicy strategy pattern
  • Lock-free store (single-writer via &mut self)
  • Lock-free load (multiple readers via &self, retry on contention)
  • Cache-line padded sequence counter (128B alignment)
  • Hardware fences for ARM64 portability
  • Multi-threaded torn-read detection test (4 readers × 500K writes)
  • Layout assertions
  • Type aliases: SeqLockDefault, SeqLockSimd (nightly)

1.11 Market State Engine (mantis-market-state)

Status: Complete | Completed: 2026-04-04

  • ArrayBook<N> — fixed-size O(1) order book indexed by tick offset
  • OrderBook trait — polymorphic interface for book implementations
  • MarketStateEngine<B, MAX> — fully stack-allocated, no_std passive state machine
  • Snapshot state machine — IS_SNAPSHOT / LAST_IN_BATCH flags for correct BBO emission
  • TopOfBook — micro price + spread + BBO cached on price change only
  • InstrumentState<B> — per-instrument state with seq, staleness, and snapshot tracking
  • Strategy trait + OrderIntent + OrderAction — inline callback hook for strategies
  • Lazy query methods: micro_price, book_imbalance, spread, last_trade, is_stale, is_ready, book
  • take_tob — edge-triggered TopOfBook emission (clears flag on read)
  • TopOfBook event processed as synthetic single-level book update
  • Layout assertions: TopOfBook fits in one 64B cache line; ArrayBook<100> size verified
  • Criterion benchmarks: apply_delta, best_bid, process_delta (mid/end), micro_price, book_imbalance
  • Bolero property tests: random deltas no panic, best_bid invariant, imbalance range
  • 17 unit tests, no_std clean, layout assertions in mantis-layout

1.12 Strategy Runtime (mantis-strategy)

Status: Complete | Completed: 2026-04-06

  • Event-driven Strategy trait — no generics, associated consts (STRATEGY_ID, NAME)
  • OrderIntent + OrderAction (Post/Cancel/Amend) with target_order_id disambiguation
  • SignedLots(i64) newtype in mantis-types for signed inventory positions
  • Position — signed qty (SignedLots), VWAP entry, realized/unrealized PnL, flat invariant
  • OrderTracker — fixed-size [Option<TrackedOrder>; 64] order state machine with slot reclamation
  • QueueEstimator — L2 probabilistic queue model (PowerProbQueueFunc, per-instrument take rates)
  • ExposureView — composes position + open orders for worst-case risk calculation
  • RiskLimits + RiskCheckResult — per-strategy risk configuration
  • StrategyContext<B, MAX> — optional helper bundle (engine + queue + orders + risk + positions)
  • Old Strategy trait removed from mantis-market-state (clean break, no deprecation)
  • 38 unit tests, no_std clean, clippy clean, 3 rounds of Codex review

1.13 Transport & Venue Decoders (mantis-transport, mantis-binance, mantis-polymarket)

Status: Complete | Completed: 2026-04-11

  • WebSocket transport layer — blocking IO on CPU-pinned threads, no async runtime
  • Polymarket market WS adapter — heartbeat, subscription, reconnect with backoff
  • Binance futures bookTicker adapter — single + multi-symbol URL generation
  • FnMut(&mut [u8]) -> bool callback for simd-json in-place parsing
  • BinanceDecoder<D> — multi-symbol (1-8) with flat lookup, combined stream peek
  • PolymarketMarketDecoder<'r, D> — book/price_change/trade with peek_type dispatch
  • Zero-alloc decode path — borrowed serde structs, parse_decimal_bytes, Ticks/Lots conversion
  • simd-json pluggable via feature flag (serde_json fallback)
  • spawn_binance_feed / spawn_polymarket_market_feed — monomorphized push callback
  • Backpressure: FnMut(HotEvent) -> bool with drop_count tracking
  • FeedSpawnResult with event_count + drop_count for monitoring
  • TimerThread — periodic Timer(Periodic) + Heartbeat events, validated config
  • FeedMonitor — stale feed detection via event_count comparison, first-check baseline
  • Timestamp::now() — wall-clock access (std feature)
  • Truncation warning for Polymarket snapshots >64 levels
  • ~307ns Binance bookTicker, ~306ns Polymarket price_change decode latency
  • Criterion benchmarks (decode.rs) + decimal parse contenders (fixed.rs)
  • 13 binance tests, 21 polymarket tests, 7 timer tests, 8 monitor tests
  • Bolero property tests: arbitrary bytes → decoders never panic
  • READMEs for all three crates

1.4 Snapshot Publication

  • Single-writer publication primitive
  • Lock-free reader access

1.5 Order Book Engine

  • Single-writer order book engine
  • Level management (add/remove/modify)
  • Top-of-book query
  • Benchmark vs naive implementations

1.6 Capture / Replay

  • Capture file format v0
  • Writer / reader implementations
  • Replay runner feeding events into engines
  • Deterministic replay validation

1.7 Tooling

  • Layout report for all hot-path structs
  • First replay diff format
  • Fuzz targets for SPSC ring + event parsing

Exit criteria:

  • Stable end-to-end: capture -> replay -> state update -> output
  • p50/p99 benchmark output exists
  • First invariant tests pass
  • First docs/examples usable by outside readers

Phase 2 — First Compelling OSS Release Candidate

Status: Not Started

  • Constant-product AMM engine MVP
  • Perp state / funding / risk primitives MVP
  • Visualizer: stage graph, queue depths, latency timeline
  • Parser/engine fuzz targets
  • Synthetic exchange replay example
  • Synthetic AMM/perp replay example
  • Queue occupancy instrumentation
  • Stage timing capture
  • Divergence report between two replay runs

Phase 3 — Fast OSS v0.1 Release

Status: Not Started

  • Documentation pass
  • Public architecture diagrams
  • Contribution guide
  • Versioning policy + changelog
  • Benchmark baseline published
  • 2-3 example applications (CEX book replay, DEX route, perp monitor)
  • README with demo GIFs/screenshots

Crate Status Summary

Crate Status no_std Tests Benchmarks Verification
mantis-core Active yes 1 — —
mantis-types Active yes 98 — —
mantis-fixed Active yes 110 7 groups + 2 contenders miri pass, 7 bolero props, 2 fuzz
mantis-events Active yes 62 — miri pass
mantis-queue Active yes 31 — miri pass
mantis-platform Active yes 164 — miri pass
mantis-seqlock Active yes 1 — miri pass
mantis-market-state Active yes 17 6 criterion groups 3 bolero props
mantis-bench Active std 11 6+7 bench groups, 6 contenders —
mantis-layout Active std 6 — —
mantis-verify Active std 13 — 4 kani proofs, 13 bolero/diff
mantis-strategy Active yes 38 — 3 rounds Codex review
mantis-transport Active std 22 — —
mantis-binance Active std 13 1 criterion group 1 bolero prop
mantis-polymarket Active std 21 4 criterion groups 1 bolero prop
mantis-registry Active std 41 — —