recorder: inline mode, so a host can have the rows without keeping the bytes

.github/workflows/recorder-release.yml

@@ -154,6 +154,14 @@ jobs:
       # afpacket needs it at build time, and the released binary carries the
       # feature: a host on a plain interface uses it, and one on a DoubleZero
       # tunnel refuses it at open rather than recording nothing.
+      #
+      # The three steps below name `afpacket` and nothing else, and features are
+      # additive, so the asset also carries the default set — which includes
+      # `inline`. That is deliberate: inline mode is what a command line naming
+      # no mode is read as, and a released binary that could not run its own
+      # default mode would refuse every ordinary command line. A host that wants
+      # the record-only binary builds it with `--no-default-features` rather
+      # than taking this asset.
       - name: Install libpcap
         run: sudo apt-get update && sudo apt-get install -y libpcap-dev
 

.github/workflows/rust-codec.yml

@@ -67,14 +67,26 @@ jobs:
         run: cargo clippy -p dz-recorder-capture --features afpacket --all-targets -- -D warnings
       - name: Test (afpacket)
         run: cargo test -p dz-recorder-capture --features afpacket
-      # The binary's own afpacket arms. Without this line every
+      # The binary's own afpacket branches. Without this line every
       # `#[cfg(feature = "afpacket")]` block in runner.rs and startup.rs is
       # neither type-checked nor run — in the *default* capture mode, which is
       # how a quiet feed that could not be shut down stayed invisible.
       - name: Clippy (dz-recorder, afpacket)
         run: cargo clippy -p dz-recorder --features afpacket --all-targets -- -D warnings
       - name: Test (dz-recorder, afpacket)
         run: cargo test -p dz-recorder --features afpacket
+      # The record-only build, and the same argument one feature over. Inline
+      # mode is the default mode, so its feature is in the default set and every
+      # other job here compiles it; this is the only job that compiles the
+      # binary without it. What lives behind `#[cfg(not(feature = "inline"))]`
+      # is the refusal a build which cannot derive a row makes when a command
+      # line names no mode — the arrangement that reading asks for — and a
+      # refusal only exercised in the builds that never make it is no refusal at
+      # all.
+      - name: Clippy (dz-recorder, no default features)
+        run: cargo clippy -p dz-recorder --no-default-features --all-targets -- -D warnings
+      - name: Test (dz-recorder, no default features)
+        run: cargo test -p dz-recorder --no-default-features
       # The live capture needs CAP_NET_RAW and a real device, so
       # afpacket-live-tests is compiled here and run by hand: a test that can
       # only run by hand must not be able to fail the build.

BRINGING-UP-A-FEED.md

@@ -394,6 +394,138 @@ Three things to know before the first run:
   into a feed-loss incident. It gives up history instead, counts what it gave
   up, and never gives up the segment it is publishing.
 
+### Pointing a feed at a mode
+
+The configuration above is **archive mode**: `dz-recorder` writes objects and
+`dz-recorder-load` derives rows from them. It is what a host recording a
+production feed for evidence runs, and **the `[archive] staging_dir` and
+`completed_dir` in that file are what select it** — there is no flag:
+
+```bash
+dz-recorder --config /etc/dz-recorder/recorder.toml
+```
+
+**Neither arrangement is a default and no flag names either.** Each is selected
+by the resource only it can run on, and the four cases are all the cases there
+are:
+
+| `[archive]` directories | `--inline-config` | What runs |
+|---|---|---|
+| stated | not given | archive mode |
+| not stated | given | inline mode |
+| stated | given | **refused**, naming the key and the file |
+| not stated | not given | **refused**, naming both |
+
+So a host cannot drift between the two: changing arrangement takes an edit that
+positively states the new one, and every half-finished edit is a non-zero exit
+code before a socket is bound. Deleting the `[archive]` section to stop keeping
+bytes for an afternoon reaches the fourth row and refuses — it does not become
+inline mode. And nothing in any unit file, pipeline or runbook needs an edit for
+this: an archive host selects its arrangement by saying what it already said.
+
+**Inline mode** is the other arrangement — one process that captures the feed,
+derives its rows and loads them, keeping no datagrams. It is for bringing a feed
+up, for a host that was never going to keep the bytes, and for one deploy unit.
+What it gives up is not small: a conformance rule written next month has nothing
+to run against, a row cannot be re-derived, nothing verified the bytes the rows
+came from, and **no market data rows are derived at all** — `event`,
+`instrument` and `book_top` come from a codec walk that archive mode runs in the
+loader. Read
+[the two modes](rust/recorder/README.md#the-two-modes) before putting a host in
+it.
+
+**Two files, and neither has a password key.** The feed above stays in the
+recorder's own file — group, ports, `expected_sources`, `site` and `recorder` —
+because that file's `config_hash` is written into every row as provenance and a
+database endpoint is not part of what a recorder does. Inline mode's own file
+carries the window bound, the ring, the spool and its budget, the ledger and the
+destination, and is passed beside it:
+
+```bash
+dz-recorder --config /etc/dz-recorder/recorder.toml \
+            --inline-config /etc/dz-recorder/inline.toml
+```
+
+```toml
+# /etc/dz-recorder/inline.toml — see rust/recorder/dz-recorder/inline.example.toml
+[inline]
+window_bytes    = "16MiB"
+window_interval = "10s"
+ring_datagrams  = 8192
+spool_dir       = "/var/lib/dz-recorder-inline/spool"
+spool_max       = "8GiB"
+ledger          = "/var/lib/dz-recorder-inline/ledger.jsonl"
+
+[clickhouse]
+endpoint = "http://192.0.2.20:8123"
+database = "recorder"
+user     = "dz_loader"
+```
+
+- **The `[archive]` section comes out.** Nothing writes an object here, so a
+  configuration stating both this file and a `staging_dir` or `completed_dir` is
+  a contradiction rather than a mode, and is refused at startup naming the key
+  and the file. Ignoring it quietly is how a host is believed to be keeping bytes
+  for a year that it never kept for a second.
+- **A configuration stating neither an archive nor this file is refused too**,
+  naming both. Inline mode needs a spool directory, a ledger and a destination,
+  archive mode needs its two directories, and there is no defensible value to
+  invent for any of the five: a recorder that guessed a destination would load
+  rows into a database nobody chose. Silence is not a mode.
+- **No `[[market_data]]` entry is accepted**, and one is refused by name rather
+  than ignored. `event`, `instrument` and `book_top` come from a codec walk, and
+  nothing in the record path decodes a datagram — the rule that makes the
+  transport grains trustworthy, since a message a decoder would reject still
+  carries the sequence number whose absence is the finding. A feed whose market
+  data rows are wanted runs archive mode, and the loader derives them.
+- **`spool_dir` must exist and be writable by the service user**, and `ledger`
+  must not be inside it. Both are refused at startup, the second for the reason
+  the loader's ledger may not live inside its objects directory: a file the
+  budget cannot classify is a file eviction cannot reach.
+- **The password comes from `DZ_LOADER_CLICKHOUSE_PASSWORD_FILE`** — a systemd
+  credential, readable by the service user alone — or from
+  `DZ_LOADER_CLICKHOUSE_PASSWORD`, and from nowhere else. That is where the
+  loader's already comes from; inline mode invents no second mechanism.
+- **The build carries the mode by default.** `inline` is a default feature,
+  because the arrangement is a property of a host's configuration and the
+  released asset is one asset for the fleet — a build carrying one arrangement
+  would have to be matched to configurations at deploy time. Add `--features tls`
+  for an `https` destination: that endpoint is refused rather than silently
+  downgraded to plain HTTP with the password on the wire. A binary built with
+  `--no-default-features` is the record-only one and can only be in archive mode,
+  so it refuses a configuration that selects inline mode by the feature's name.
+
+**`--check` is what the deployment pipeline runs before it restarts anything**,
+in either mode, and it is an `ExecStartPre` in the units:
+
+```bash
+dz-recorder --config /etc/dz-recorder/recorder.toml --check
+dz-recorder --config /etc/dz-recorder/recorder.toml \
+            --inline-config /etc/dz-recorder/inline.toml --check
+```
+
+It validates both files, plans the feeds — every refusal archive mode makes
+about a group, a port role or an interface is made here too — prints which
+arrangement is running and what it keeps, and asks the destination for
+`SELECT 1`. **The arrangement is the first line of its output in both modes**, so
+a deployment pipeline can read which one it has just deployed rather than infer
+it from which keys were echoed back. It is also where a configuration stating
+both arrangements, or neither, is caught: run as an `ExecStartPre`, an unclear
+arrangement costs a failed pre-check rather than a recorder that will not
+start. Nothing is bound, nothing is created and nothing is joined, and the
+spool and the ledger are not touched, so it is safe against a host that is
+already recording. A gate that passed without reaching the destination would let
+a pipeline restart a recorder that cannot write.
+
+**Alert on the age of the oldest unposted window, and never on the eviction
+counter.** A full spool budget evicts on every window at steady state by design,
+so the counter rises whether or not anything is wrong; one window older than the
+eviction horizon is history already gone, and no re-run recovers it. This is the
+same rule as
+[`dz_loader_oldest_unloaded_age_seconds`](rust/recorder/dz-recorder-load/README.md#the-gate-on-that-arrangement)
+in archive mode, over windows instead of objects. Size `spool_max` against the
+column-store outage the host intends to survive.
+
 ### Auditing what was recorded
 
 The point of keeping the bytes is answering *did the publisher send what the
@@ -456,6 +588,7 @@ infrastructure repositories, and each of those owns its own review.
 - [ ] config reviewed for `pin`, `source_id`, `channel_id`, group and ports
 - [ ] for a depth feed: `snapshot_cycle` set, and the adapter's `DepthBound` checked against what its book actually holds
 - [ ] a recorder is configured for the feed before the publisher is pointed at production
+- [ ] the recorder's arrangement is a decision, and the configuration states it: the two `[archive]` directories for a feed being recorded for evidence, `--inline-config` only where nobody was going to keep the bytes. `--check` prints which one on its first line, and refuses if the configuration states both or neither. In inline mode, the alert on the age of the oldest unposted window exists, and nothing on this host expects `event`, `instrument` or `book_top` rows
 - [ ] metrics scraped, and the connection-state alert exists
 
 **A new feed type**, additionally

docs/README.md

@@ -27,6 +27,14 @@ Names we do not own, or have not renamed, are left as written wherever they appe
 | Market-by-Order | [demo stack](2026-04-23-marketbyorder-design.md), [rename from depth-of-book](superpowers/specs/2026-06-05-marketbyorder-rename-design.md), [snapshot resilience](superpowers/specs/2026-06-06-marketbyorder-bot-snapshot-resilience-design.md), [shard dispatcher](2026-05-19-marketbyorder-bot-shard-dispatcher-design.md) | [demo stack](2026-04-23-marketbyorder-plan.md), [rename](superpowers/plans/2026-06-05-marketbyorder-rename.md), [snapshot resilience](superpowers/plans/2026-06-06-marketbyorder-bot-snapshot-resilience.md), [shard dispatcher](2026-05-19-marketbyorder-bot-shard-dispatcher-plan.md) |
 | Market-by-Price | [parser, book-builder, demo stack](superpowers/specs/2026-08-02-marketbyprice-design.md), [persistence](superpowers/specs/2026-08-07-marketbyprice-bot-persistence-design.md) | [parser](superpowers/plans/2026-08-02-marketbyprice-parser.md), [book engine](superpowers/plans/2026-08-02-marketbyprice-bot-engine.md), [persistence](superpowers/plans/2026-08-07-marketbyprice-bot-persistence.md) |
 
+## Recorder
+
+| | |
+|---|---|
+| [A generic recorder](superpowers/specs/2026-08-28-edge-recorder-crates-design.md) · [record path plan](superpowers/plans/2026-08-30-edge-recorder-record-path.md) | The design behind [`rust/recorder`](../rust/recorder/): what an archive has to hold for a gap in it to be attributable |
+| [Sequence loss and conformance rows](superpowers/specs/2026-08-31-sequence-loss-and-conformance-rows-design.md) · [plan](superpowers/plans/2026-09-03-recorder-clickhouse-rows.md) | Which sequence values nobody delivered, whose they are, and the rows an archive derives into |
+| [Inline mode](superpowers/specs/2026-09-08-recorder-inline-mode-design.md) · [plan](superpowers/plans/2026-09-08-recorder-inline-mode.md) | One process from capture to rows, keeping no datagrams; why it is what a command line naming no mode is read as, and what that default costs |
+
 ## Cross-cutting
 
 | | |