Skip to content

Latest commit

 

History

History
65 lines (47 loc) · 7.87 KB

File metadata and controls

65 lines (47 loc) · 7.87 KB

Documents

Design documents and implementation plans, kept as a record. Each was written against the code as it stood at its date and is not updated afterwards — for current behaviour, read the component's README. On wire format, edge-feed-spec wins over anything here.

How-to material does not live here, for the reason above: a guide has to stay true. BRINGING-UP-A-FEED.md is the one for standing a feed up end to end.

superpowers/specs/ and superpowers/plans/ are the current convention: a spec argues a design and is reviewed before code, a plan turns it into ordered tasks, and they pair by name and date. The dated files at this level predate that split.

Older documents predate GLOSSARY.md. Their prose now follows it. Fenced code blocks do not, and are left as written: a block in a dated document is a record of the code, the command or the metric name as it stood that day, and rewriting it would make the document describe a tree that never existed. So a fence may still say frame where the prose beside it says datagram, and a curl line may still grep dz_mbp_parser_frames_total where the metric is now dz_mbp_parser_datagrams_total. The same holds for a metric named in prose in a document dated before the rename.

One word does not get that exemption. arm is banned by the glossary "in every sense and every place: specs, docs, plans, identifiers, CLI flags, config keys, metric names, log fields, and code comments alike", so a fenced Go comment saying re-arm was changed to say what it does — reset a window — while the rearm identifier in the same block was left alone, being real Go that the plan's own code still declares.

Names we do not own, or have not renamed, are left as written wherever they appear: Frame Length, which is the wire spec's own field name; and bot, which is still the name of the three Go module directories, the demo's compose service, and the dz_mbo_bot / dz_mbp_bot metric namespaces. Prose calls them book-builders.

Publisher crates

Shared publisher crates The design behind rust/codec and rust/publisher
Codec crates: Top-of-Book path Plan for the first three codec crates
The venue adapter interface · plan The trait a venue repository implements to turn its own upstream source into our messages, and how the recorder re-lowers it to compare against multicast
Several channel instances of one feed specification · plan One process operating many channels of one feed spec: a shard on [[feed]], reference data per channel, and the era keyed on the channel instance
An upstream write that is not a connect · plan Why an instrument admitted mid-session was never subscribed, and why the write has to name the connection it goes out on
A polled transport · plan [ingress] kind = "poll": a catalogue that is a request rather than a subscription, why the token is not rest, and why an unchanged response is not a payload
A session transport · plan [ingress] kind = "fix": what a session layer owns, why the logon body is the adapter's, and why the sequence resets

Feeds

Feed Design Plan
Market-by-Order demo stack, rename from depth-of-book, snapshot resilience, shard dispatcher demo stack, rename, snapshot resilience, shard dispatcher
Market-by-Price parser, book-builder, demo stack, persistence parser, book engine, persistence

Recorder

A generic recorder · record path plan The design behind rust/recorder: what an archive has to hold for a gap in it to be attributable
Sequence loss and conformance rows · plan Which sequence values nobody delivered, whose they are, and the rows an archive derives into
Inline mode · plan One process from capture to rows, keeping no datagrams; why it is what a command line naming no mode is read as, and what that default costs

Cross-cutting

Dual-version refdata · plan Decoding InstrumentDefinition at schema versions 1 and 3, and watching a cutover
Per-publisher sequence tracking · plan Why gap detection keys on (source IP address, Channel ID, destination port)
Cross-feed latency normalization · plan Comparing latency across feeds that timestamp differently
A key a venue side can compute · plan book_key over a book alone, a venue's own message identity at the boundary, and the venue composition seam moved out of the publisher's crate
The venue half of a feed race · plan Recording a venue's own upstream and pairing it against a channel's datagrams — why the join is a view on the symbol and not a row in book_top

Shred receivers

Rust kernel-socket receiver · plan
Rust XDP receiver · plan
Go receivers Kernel-socket and XDP
XDP GRE decapsulator The gre-decap program

Other

Receiving the Hyperliquid feed Consuming one venue's feed over DoubleZero Edge