diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 399f446..442da60 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -39,6 +39,9 @@ jobs: - name: Protocol benchmark (smoke) run: dotnet run --project Bench/Bench.csproj -c Release --no-build -- protocol --iterations 10000 --trials 3 + - name: Durability benchmark (smoke) + run: dotnet run --project Bench/Bench.csproj -c Release --no-build -- durability --records 100 --payload 64 --trials 3 --range-queries 10 + - name: Feed smoke test run: ./scripts/smoke.sh diff --git a/BENCHMARKS.md b/BENCHMARKS.md index 3947177..933763f 100644 --- a/BENCHMARKS.md +++ b/BENCHMARKS.md @@ -32,6 +32,8 @@ this host; they are not hardware-independent capacity claims. - Allocation uses `GC.GetAllocatedBytesForCurrentThread` on single-threaded paths and `GC.GetTotalAllocatedBytes` on concurrent queue paths. - Packet bytes and checksums are consumed so dead-code elimination cannot remove protocol work. +- WAL timings stop at the policy's acknowledgement point; final disposal sync is outside the timed + region. Range and recovery trials are warm-cache filesystem measurements. - Transport load is open-loop. Source timestamps precede dissemination, so backlog increases measured latency instead of reducing offered load. - CI runs smoke-sized benchmarks for rot detection; it does not gate performance on shared runners. @@ -59,6 +61,41 @@ end-to-end case includes seal, CRC validation, decoder locking, bounded reorder retention, sequencing, and in-place depth application. CRC correctness is checked against the standard `123456789` vector. Corruption tests assert that state and sequence do not advance. +## Durable publication and recovery + +Artifact: `bench/results/durability-v2.json`. Five trials; 5,000 append acknowledgements per trial, +50,000-message recovery log, and 100 ten-message range requests. The JSON embeds runtime and host +metadata. + + +| Append contract | Policy | Payload | Median | Min–max | Rate | Syncs/trial | Allocation | +|---|---|---|---|---|---|---|---| +| OS page cache | OsBuffered | 64 B | 976.7 ns | 958.0–2,088.2 ns | 1,023,815/s | 0 | 0 B/op | +| periodic 1 ms | SyncPeriodic | 64 B | 2,318.3 ns | 2,149.5–3,507.9 ns | 431,343/s | 4 | 0 B/op | +| group commit 64 | OsBuffered | 64 B | 27,301.7 ns | 17,837.9–42,412.7 ns | 36,628/s | 79 | 0 B/op | +| fsync each | SyncEachRecord | 64 B | 972,646.2 ns | 898,242.4–1,196,500.7 ns | 1,028/s | 5,000 | 0 B/op | +| seal + packet WAL | OsBuffered | 50 B | 813.7 ns | 755.1–1,676.6 ns | 1,228,894/s | 0 | 0 B/op | + +| Messages | Checkpoint | Full replay | Checkpoint + tail | Speed-up | +|---|---|---|---|---| +| 50,000 | 47,500 | 24.68 ms (21.00–40.83) | 2.92 ms (2.64–3.84) | 8.45× | + +| 10-message range | Queries | Index entries | Median | Min–max | Allocation | +|---|---|---|---|---|---| +| sparse index | 100 | 196 | 73.6 µs | 70.2–81.7 µs | 1,736 B/request | +| segment scan | 100 | 0 | 1,946.8 µs | 927.1–2,530.3 µs | 5,848 B/request | + + +`OS page cache` includes framing, CRC-32C, and one unbuffered managed write into the kernel cache; +it is not power-loss durability. `seal + packet WAL` also seals and validates the 50-byte feed +packet. The 1 ms periodic case stresses group sync; the configurable server default is 200 ms. +`fsync each` measures this virtual disk, not a portable storage latency. + +Recovery trials alternate full-first and checkpoint-first order. The checkpoint is at sequence +47,500; complete segments before it are skipped. The sparse range index stores one entry per 256 +records and incrementally follows the live tail. Both range cases copy the same ten payloads; the +table isolates lookup strategy. + ## Matching engine Artifact: `bench/results/matching-v2.json`. Each value is the median over 200,000 state-preserving @@ -168,15 +205,8 @@ arbitration with zero sequence gaps. ## Transport scaling (pre-v2 generation) -The sections above measure the v2 protocol and its recovery path. They do not measure how the -dissemination architecture scales with the *audience*, which is a separate question and the one -this project was originally built to answer. That record is retained here. - -**Read this section under the artifact boundary below.** These runs predate protocol v2 and were -recorded on a different host (4 vCPU; see the table below) from the v2 measurements (8 logical -processors). Nothing here is comparable with a v2 number, and none of it is combined into a single -claim with one. What it is comparable with is itself: every row below was measured on one host in -one session, which is what makes the unicast-versus-multicast comparison meaningful. +These runs predate protocol v2 and use a separate 4-vCPU host. Compare rows within this section; +do not compare them with v2 results. | | | @@ -190,11 +220,10 @@ one session, which is what makes the unicast-versus-multicast comparison meaning | Topology | server and load generator as separate processes on the same host | -### The O(N) wall +### Unicast fan-out -TCP fan-out performs one write per subscriber per update, so a subscriber's latency is essentially -its position in that span. Two runs matched on messages per second, differing only in how many -subscribers the work is spread across: +TCP fan-out performs one write per subscriber per update. These runs have similar delivered rates +but different subscriber counts: | Subscribers | Feed rate | Fan-out | Mean latency | @@ -203,8 +232,8 @@ subscribers the work is spread across: | 1,000 | 10 upd/s | 10,100 msg/s | **18.63 ms** | -Identical work per second; ten times the audience costs an order of magnitude more latency, and -neither run was CPU-starved. No amount of tuning removes an O(N) term. +At approximately 10,000 messages/s, increasing the audience from 100 to 1,000 subscribers raised +mean latency from 1.61 ms to 18.63 ms. The full sweep, feed rate held at 100 updates/s aggregate: @@ -222,10 +251,8 @@ The full sweep, feed rate held at 100 updates/s aggregate: | 900 | 86,621 | **46.15** | 23.75 | 216.7 | 254.2 | 322.8 | 99.2% | 97% | 229.4% | 378.8% | yes | -Every point sustained, so this sweep does not contain unicast's breaking point — but the wall is -visible in the server's own CPU, which rises from 53% to 229% of 400% while host CPU reaches 379%. -At 900 subscribers the latency distribution has begun to come apart: p99 of 216.7 ms against a mean -of 46.2 ms. +Every point sustained; 900 subscribers is the top of the sweep, not a measured limit. Server CPU +rose from 52.8% to 229.4%; p99 reached 216.7 ms at 900 subscribers. A second sweep holds the message rate constant on a lighter feed: @@ -239,7 +266,7 @@ A second sweep holds the message rate constant on a lighter feed: | 5,000 | 50,334 | **96.83** | 87.95 | 321.1 | 387.1 | 422.2 | 100.7% | 100% | 157.4% | 304.2% | yes | -### Multicast removes the term +### Multicast fan-out The publisher encodes each update once and sends a single datagram; the network performs the replication. @@ -257,21 +284,14 @@ replication. | 8,000 | 594,357 | **744.52** | 350.45 | 10365.0 | 24032.8 | 91.2% | 81.5 | 594 | 0 | 91.6% | 373.9% | **NO** | -**The `Server pkt/s` column is the result.** It does not move — 98.6 to 100.3 packets per second -from 100 subscribers to 6,000. The publisher transmits at the update rate and holds no subscriber -table at all. - -8,000 is where it breaks, and it breaks the way an unreliable transport should: 594 sequence gaps, -detected and reported by the affected subscribers rather than silently corrupting anybody's book. -That failure is left in the table rather than trimmed off the end of it. +Server packet rate stayed between 98.6 and 100.3/s through 6,000 subscribers. The 8,000-subscriber +run was not sustained and recorded 594 sequence gaps. | Subscribers | Unicast mean | Multicast mean | Improvement | |---|---|---|---| | 100 | 1.61 ms | **0.31 ms** | **5.3×** | | 500 | 8.54 ms | **0.85 ms** | **10.1×** | - -Multicast was also measured at 250, 1,000, 2,000, 4,000, 6,000, 8,000 subscribers, where unicast was not run; those points are in the multicast sweep in BENCHMARKS.md. Cost per delivered message, at each transport's highest sustained point: @@ -281,19 +301,14 @@ Cost per delivered message, at each transport's highest sustained point: |---|---|---|---|---| | Unicast gRPC | 900 | 86,621 | 229.4% | **26.48 µs** | | Multicast | 6,000 | 594,067 | 73.2% | **1.23 µs** | - -Multicast delivers each message for **21× less server CPU**, to **6.7× the subscribers** at **6.9× the throughput**. -Note the asymmetry in that table: multicast's 6,000 is a real ceiling because 8,000 was measured -and failed, whereas unicast's 900 is simply the top of the sweep — no unicast point failed, so its -limit was never found and lies somewhere above 900. +The next multicast point failed; no unicast failure point was measured. ### Batching -Batching normally trades latency for throughput. On a fan-out feed it does the opposite, because -per-packet cost is paid once per *subscriber*: 1,000 subscribers, 1,000 updates/s aggregate, -varying only how many messages the publisher packs into a datagram. +The following runs use 1,000 subscribers and 1,000 aggregate updates/s while varying packet batch +size. | Max batch | Fan-out (msg/s) | Mean (ms) | p99 | Server pkt/s | Server CPU | Host CPU | @@ -304,10 +319,10 @@ varying only how many messages the publisher packs into a datagram. | 64 | 971,792 | **2.09** | 4.7 | 201.0 | 45.0% | 170.9% | -Packet rate falls 4.4×, mean latency 1.6×, p99 3.1×, host CPU 2.2× — while delivered throughput -*rises* 12%. +Batch 64 versus batch 1 reduced packet rate 4.4×, mean latency 1.6×, p99 3.1×, and host CPU 2.2×; +delivered throughput increased 12%. -### Repeatability, and what a single sweep point is worth +### Repeatability | Point | Runs | Median | Min | Max | Spread | @@ -316,8 +331,7 @@ Packet rate falls 4.4×, mean latency 1.6×, p99 3.1×, host CPU 2.2× — while | 500 subscribers, 100 upd/s | 3 | 8.64 ms | 7.73 | 8.66 | 1.12× | -A 1.4× range across three runs of an identical configuration at 4,000 subscribers. Every point in -the sweep tables above is a single run, so read them with that spread in mind. +The 4,000-subscriber configuration varied 1.4× across three runs. Sweep rows are single runs. ## Market realism @@ -344,11 +358,8 @@ Files carrying `v2` or `protocolv2` are the current protocol/recovery record. Un the earlier transport sweeps, microstructure study, regenerated controls, and repeatability runs, presented under [Transport scaling](#transport-scaling-pre-v2-generation). -The two generations were measured on different hosts and **are not combined into one claim**. -`docgen.py` enforces this: it partitions results by generation and refuses to render if either -generation contains results from more than one kernel instance. What that check cannot enforce is -prose, so the rule for a reader is simple — a v2 number and a pre-v2 number never belong in the same -sentence, and no comparison in this document crosses that line. +The generations use different hosts and are not compared. `docgen.py` rejects mixed kernel +instances within either generation. ## Reproduce @@ -358,6 +369,10 @@ dotnet build MarketDataSimulator.sln -c Release dotnet run --project Bench -c Release --no-build -- \ protocol --iterations 1000000 --trials 7 --out bench/results/protocol-v2.json +dotnet run --project Bench -c Release --no-build -- \ + durability --records 5000 --payload 64 --trials 5 --range-queries 100 \ + --out bench/results/durability-v2.json + dotnet run --project Bench -c Release --no-build -- \ queue --items 1000000 --capacity 8192 --trials 7 --out bench/results/queue-v2.json @@ -375,9 +390,7 @@ python3 bench/docgen.py --write python3 bench/docgen.py --check ``` -The pre-v2 transport generation was produced by the sweeps below. Re-running them re-measures that -whole generation on the current host, which is the only correct way to refresh it — the guard will -reject a partial refresh that mixes hosts within one generation. +Refresh the complete pre-v2 generation together; mixed-host partial refreshes are rejected. ```bash python3 bench/environment.py @@ -400,6 +413,6 @@ for i in 1 2 3; do done ``` -A serious capacity study should reserve hosts, pin processes and interrupts, record frequency and -thermal state, separate publishers and consumers, inject controlled loss/reordering, capture -hardware counters, and repeat across x64 and Arm64. +A capacity study requires reserved hosts, process and interrupt affinity, frequency and thermal +telemetry, separate publishers and consumers, controlled loss/reordering, hardware counters, and +x64/Arm64 runs. diff --git a/Bench/DurabilityBenchmark.cs b/Bench/DurabilityBenchmark.cs index c0b24bc..bd4c8bf 100644 --- a/Bench/DurabilityBenchmark.cs +++ b/Bench/DurabilityBenchmark.cs @@ -1,31 +1,30 @@ -using MarketData.Common.Books; -using MarketData.Common.Durability; using System; +using System.Buffers.Binary; using System.Collections.Generic; using System.Diagnostics; using System.Globalization; using System.IO; using System.Linq; +using System.Runtime; +using System.Runtime.InteropServices; using System.Text.Json; +using MarketData.Common.Books; +using MarketData.Common.Durability; +using MarketData.Common.Feed; namespace MarketData.Bench { - /// - /// What durability costs, and what recovery costs. - /// - /// - /// Both numbers exist to make a design decision falsifiable rather than asserted. "Journal - /// every message before publishing it" is only defensible if the append is cheap next to the - /// publish; "checkpoint periodically" is only worth the machinery if recovery time actually - /// stops growing with uptime. Neither is obvious, so both are measured. - /// + /// WAL acknowledgement, replay, checkpoint, and range-read costs. public static class DurabilityBenchmark { + private const ulong Session = 0xD0A8_1EUL; + public static int Run(string[] args) { - var records = 50_000; + var records = 10_000; var payloadBytes = 64; var trials = 5; + var rangeQueries = 200; string outputPath = null; for (var i = 0; i < args.Length; i++) @@ -35,216 +34,394 @@ public static int Run(string[] args) case "--records": records = int.Parse(args[++i], CultureInfo.InvariantCulture); break; case "--payload": payloadBytes = int.Parse(args[++i], CultureInfo.InvariantCulture); break; case "--trials": trials = int.Parse(args[++i], CultureInfo.InvariantCulture); break; + case "--range-queries": rangeQueries = int.Parse(args[++i], CultureInfo.InvariantCulture); break; case "--out": outputPath = args[++i]; break; + default: throw new ArgumentException($"Unknown durability option: {args[i]}"); } } - Console.WriteLine($"Durability benchmark: {records:N0} records of {payloadBytes} B, " + - $"median of {trials} trials"); - Console.WriteLine(); + if (records is < 100 or > 10_000_000 || payloadBytes < 0 || + payloadBytes > JournalRecord.MaxPayloadSize || trials is < 3 or > 101 || + rangeQueries is < 10 or > 1_000_000) + throw new ArgumentOutOfRangeException(nameof(args)); - var root = Path.Combine(Path.GetTempPath(), "mds-bench-" + Guid.NewGuid().ToString("N")); + var root = Path.Combine(Path.GetTempPath(), "mds-durability-" + Guid.NewGuid().ToString("N")); Directory.CreateDirectory(root); - var results = new List(); - try { - Console.WriteLine($"{"Durability policy",22} {"ns/append",12} {"appends/s",14} {"syncs",10} {"MB/s",9}"); - Console.WriteLine(new string('-', 72)); - - foreach (var policy in new[] - { - DurabilityPolicy.OsBuffered, - DurabilityPolicy.SyncPeriodic, - DurabilityPolicy.SyncEachRecord, - }) + var append = MeasureAppendCases(root, records, payloadBytes, trials); + var recovery = MeasureRecovery(root, Math.Max(records, 50_000), trials); + var ranges = MeasureRanges(root, Math.Max(records, 50_000), rangeQueries, trials); + + Print(append, recovery, ranges); + + var report = new DurabilityReport( + DateTimeOffset.UtcNow, + RuntimeInformation.FrameworkDescription, + RuntimeInformation.OSDescription, + RuntimeInformation.ProcessArchitecture.ToString(), + Environment.ProcessorCount, + GCSettings.IsServerGC, + Stopwatch.Frequency, + Crc32C.Implementation, + records, + payloadBytes, + trials, + append, + recovery, + ranges); + + if (outputPath is not null) { - var samples = new List(); - long syncs = 0; + Directory.CreateDirectory(Path.GetDirectoryName(Path.GetFullPath(outputPath))!); + File.WriteAllText(outputPath, JsonSerializer.Serialize(report, + new JsonSerializerOptions { WriteIndented = true }) + Environment.NewLine); + Console.WriteLine($"Wrote {outputPath}"); + } - for (var trial = 0; trial < trials + 1; trial++) - { - var directory = Path.Combine(root, $"{policy}-{trial}"); - var payload = new byte[payloadBytes]; - var stopwatch = Stopwatch.StartNew(); + return 0; + } + finally + { + try { Directory.Delete(root, recursive: true); } catch (IOException) { } + } + } - using (var journal = new WriteAheadJournal(directory, 1, policy)) - { - for (var i = 1; i <= records; i++) - journal.Append(JournalRecordType.Message, (ulong)i, i, payload); + private static AppendResult[] MeasureAppendCases(string root, int records, int payloadBytes, + int trials) + { + var cases = new[] + { + new AppendCase("OS page cache", DurabilityPolicy.OsBuffered, TimeSpan.FromMilliseconds(200), 0), + new AppendCase("periodic 1 ms", DurabilityPolicy.SyncPeriodic, TimeSpan.FromMilliseconds(1), 0), + new AppendCase("group commit 64", DurabilityPolicy.OsBuffered, TimeSpan.FromMilliseconds(200), 64), + new AppendCase("fsync each", DurabilityPolicy.SyncEachRecord, TimeSpan.FromMilliseconds(200), 0), + }; + + var results = new List(cases.Length + 1); + foreach (var item in cases) + results.Add(MeasureAppend(root, item, records, payloadBytes, trials)); + results.Add(MeasureFeedPacketAppend(root, records, trials)); + return results.ToArray(); + } - stopwatch.Stop(); - syncs = journal.Syncs; - } + private static AppendResult MeasureAppend(string root, AppendCase item, int records, + int payloadBytes, int trials) + { + var elapsed = new double[trials]; + var allocated = new double[trials]; + var syncs = new long[trials]; + var payload = new byte[payloadBytes]; - // First trial is warm-up: it pays for JIT and for creating the directory. - if (trial > 0) - samples.Add(stopwatch.Elapsed.TotalNanoseconds / records); + for (var trial = -1; trial < trials; trial++) + { + var directory = Path.Combine(root, $"append-{item.Name.Replace(' ', '-')}-{trial}"); + using (var journal = new WriteAheadJournal(directory, Session, item.Policy, + syncInterval: item.Interval)) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + var started = Stopwatch.GetTimestamp(); - Directory.Delete(directory, recursive: true); + for (var sequence = 1; sequence <= records; sequence++) + { + journal.Append(JournalRecordType.Message, (ulong)sequence, sequence, payload); + if (item.GroupSize != 0 && sequence % item.GroupSize == 0) + journal.Sync(); } - var median = Median(samples); - var perSecond = 1_000_000_000.0 / median; - var bytesPerSecond = perSecond * JournalRecord.SizeFor(payloadBytes); + if (item.GroupSize != 0 && records % item.GroupSize != 0) + journal.Sync(); - var elapsedMs = median * records / 1_000_000.0; + var ticks = Stopwatch.GetTimestamp() - started; + var bytes = GC.GetAllocatedBytesForCurrentThread() - before; - // A periodic policy that never reached its interval synced zero times, and - // reporting that without saying why invites the reader to conclude it never - // syncs at all. State the run length against the interval instead. - var note = policy == DurabilityPolicy.SyncPeriodic && syncs == 0 - ? $" (run took {elapsedMs:N0} ms, under the {WriteAheadJournal.FlushInterval.TotalMilliseconds:N0} ms interval, so no periodic sync fell due)" - : string.Empty; - - Console.WriteLine($"{policy,22} {median,12:N1} {perSecond,14:N0} {syncs,10:N0} " + - $"{bytesPerSecond / 1024 / 1024,9:N1}{note}"); - - results.Add(new + if (trial >= 0) { - Kind = "append", - Policy = policy.ToString(), - PayloadBytes = payloadBytes, - NanosecondsPerAppend = Math.Round(median, 1), - AppendsPerSecond = Math.Round(perSecond, 0), - Syncs = syncs, - RunMilliseconds = Math.Round(median * records / 1_000_000.0, 1), - FlushIntervalMilliseconds = WriteAheadJournal.FlushInterval.TotalMilliseconds, - MegabytesPerSecond = Math.Round(bytesPerSecond / 1024 / 1024, 1), - }); + elapsed[trial] = ticks * (1_000_000_000.0 / Stopwatch.Frequency) / records; + allocated[trial] = bytes / (double)records; + syncs[trial] = journal.Syncs; + } } - Console.WriteLine(); - Console.WriteLine("Recovery: rebuilding book state from a journal, with and without a checkpoint."); - Console.WriteLine(); - Console.WriteLine($"{"Journalled messages",20} {"full replay ms",16} {"from checkpoint ms",20} {"speed-up",10}"); - Console.WriteLine(new string('-', 72)); + Directory.Delete(directory, recursive: true); + } + + return SummariseAppend(item.Name, payloadBytes, records, elapsed, allocated, syncs, + item.Policy, item.Interval.TotalMilliseconds); + } - foreach (var count in new[] { 10_000, 50_000, 200_000 }) + private static AppendResult MeasureFeedPacketAppend(string root, int records, int trials) + { + var elapsed = new double[trials]; + var allocated = new double[trials]; + var syncs = new long[trials]; + var packet = new byte[FeedProtocol.HeaderSize + FeedProtocol.IncrementalSize]; + FeedProtocol.WriteIncremental(packet.AsSpan(FeedProtocol.HeaderSize), FeedMessageType.Add, + 1, Side.Bid, new PriceLevel(-1, 100)); + + for (var trial = -1; trial < trials; trial++) + { + var directory = Path.Combine(root, $"feed-packet-{trial}"); + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered, initialSequence: 0)) { - var directory = Path.Combine(root, $"recover-{count}"); - var checkpoints = Path.Combine(root, $"recover-{count}-chk"); - var random = new Random(count); + var before = GC.GetAllocatedBytesForCurrentThread(); + var started = Stopwatch.GetTimestamp(); - IOrderBook Make(int _) => new SortedArrayBook(16); - var books = new Dictionary(); + for (var sequence = 0; sequence < records; sequence++) + { + FeedProtocol.WriteHeader(packet, 1, Session, (ulong)sequence, sequence); + journal.AppendPacket(packet); + } - // Small segments so the log actually rotates, which is what production does and - // what lets recovery skip history below the checkpoint. With one giant segment - // there is nothing to skip and a checkpoint saves only the book replay, not the - // scan - which is exactly the trap the first version of this benchmark fell into. - const long segmentBytes = JournalRecord.OverheadSize + JournalRecord.MaxPayloadSize; + var ticks = Stopwatch.GetTimestamp() - started; + var bytes = GC.GetAllocatedBytesForCurrentThread() - before; - using (var journal = new WriteAheadJournal(directory, 1, DurabilityPolicy.OsBuffered, - segmentBytes)) + if (trial >= 0) { - for (var i = 1; i <= count; i++) - { - var instrument = random.Next(1, 4); - var side = random.Next(2) == 0 ? Side.Bid : Side.Ask; - var price = random.Next(-50, 51); - var quantity = (uint)random.Next(0, 500); - - if (!books.TryGetValue(instrument, out var book)) - books[instrument] = book = Make(instrument); - - book.Upsert(side, price, quantity); - - var encoded = new byte[13]; - System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(encoded, instrument); - encoded[4] = (byte)side; - System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(encoded.AsSpan(5), price); - System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(encoded.AsSpan(9), quantity); - - journal.Append(JournalRecordType.Message, (ulong)i, i, encoded); - - // One checkpoint near the end, which is the realistic case: recovery - // replays only what has happened since the last one. - if (i == count - count / 20) - Checkpoint.Write(checkpoints, journal, (ulong)i, 1, books); - } + elapsed[trial] = ticks * (1_000_000_000.0 / Stopwatch.Frequency) / records; + allocated[trial] = bytes / (double)records; + syncs[trial] = journal.Syncs; } + } - var full = TimeRecovery(directory, null, Make); - var incremental = TimeRecovery(directory, Checkpoint.FindLatest(checkpoints), Make); + Directory.Delete(directory, recursive: true); + } - Console.WriteLine($"{count,20:N0} {full,16:N1} {incremental,20:N1} " + - $"{full / Math.Max(incremental, 0.0001),9:N1}x"); + return SummariseAppend("seal + packet WAL", packet.Length, records, elapsed, allocated, + syncs, DurabilityPolicy.OsBuffered, 0); + } - results.Add(new - { - Kind = "recovery", - Messages = count, - FullReplayMilliseconds = Math.Round(full, 1), - FromCheckpointMilliseconds = Math.Round(incremental, 1), - SpeedUp = Math.Round(full / Math.Max(incremental, 0.0001), 1), - }); - - Directory.Delete(directory, recursive: true); - if (Directory.Exists(checkpoints)) - Directory.Delete(checkpoints, recursive: true); + private static RecoveryResult MeasureRecovery(string root, int messages, int trials) + { + var directory = Path.Combine(root, "recovery-journal"); + var checkpoints = Path.Combine(root, "recovery-checkpoints"); + var random = new Random(20260819); + var books = new Dictionary(); + var encoded = new byte[13]; + var checkpointSequence = messages - messages / 20; + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered, segmentBytes)) + { + for (var sequence = 1; sequence <= messages; sequence++) + { + var instrument = random.Next(1, 4); + var side = random.Next(2) == 0 ? Side.Bid : Side.Ask; + var price = random.Next(-50, 51); + var quantity = (uint)random.Next(0, 500); + + if (!books.TryGetValue(instrument, out var book)) + books[instrument] = book = new SortedArrayBook(16); + book.Upsert(side, price, quantity); + + BinaryPrimitives.WriteInt32LittleEndian(encoded, instrument); + encoded[4] = (byte)side; + BinaryPrimitives.WriteInt32LittleEndian(encoded.AsSpan(5), price); + BinaryPrimitives.WriteUInt32LittleEndian(encoded.AsSpan(9), quantity); + journal.Append(JournalRecordType.Message, (ulong)sequence, sequence, encoded); + + if (sequence == checkpointSequence) + Checkpoint.Write(checkpoints, journal, (ulong)sequence, Session, books); } + } - Console.WriteLine(); - Console.WriteLine("Recovery from a checkpoint is bounded by messages since the checkpoint,"); - Console.WriteLine("not by total uptime. That is the entire reason checkpoints exist."); + var checkpoint = Checkpoint.FindLatest(checkpoints)!; + _ = TimeRecovery(directory, null); + _ = TimeRecovery(directory, checkpoint); + var full = new double[trials]; + var incremental = new double[trials]; - if (outputPath is not null) + for (var trial = 0; trial < trials; trial++) + { + if ((trial & 1) == 0) { - var directory = Path.GetDirectoryName(Path.GetFullPath(outputPath)); - Directory.CreateDirectory(directory); - File.WriteAllText(outputPath, - JsonSerializer.Serialize(results, new JsonSerializerOptions { WriteIndented = true })); - Console.WriteLine($"Wrote {outputPath}"); + full[trial] = TimeRecovery(directory, null); + incremental[trial] = TimeRecovery(directory, checkpoint); + } + else + { + incremental[trial] = TimeRecovery(directory, checkpoint); + full[trial] = TimeRecovery(directory, null); } } - finally + + Array.Sort(full); + Array.Sort(incremental); + return new RecoveryResult(messages, checkpointSequence, + Round(full[trials / 2]), Round(full[0]), Round(full[^1]), + Round(incremental[trials / 2]), Round(incremental[0]), Round(incremental[^1]), + Math.Round(full[trials / 2] / incremental[trials / 2], 2)); + } + + private static RangeResult[] MeasureRanges(string root, int messages, int queries, int trials) + { + var directory = Path.Combine(root, "range-journal"); + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + var payload = new byte[64]; + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered, segmentBytes)) { - try { Directory.Delete(root, recursive: true); } catch (IOException) { } + for (var sequence = 1; sequence <= messages; sequence++) + journal.Append(JournalRecordType.Message, (ulong)sequence, sequence, payload); } - return 0; + var index = new JournalRangeReader(directory); + var starts = new ulong[queries]; + var random = new Random(17); + for (var i = 0; i < starts.Length; i++) + starts[i] = (ulong)random.Next(1, messages - 10); + + return new[] + { + MeasureRanges("sparse index", queries, trials, starts, + (ulong from, ulong to, out List found) => + index.TryRead(Session, from, to, out found), index.IndexEntries), + MeasureRanges("segment scan", queries, trials, starts, + (ulong from, ulong to, out List found) => + JournalReader.TryReadRange(directory, Session, from, to, out found), 0), + }; } - private static double TimeRecovery(string directory, string checkpointPath, Func make) + private static RangeResult MeasureRanges(string name, int queries, int trials, + ulong[] starts, RangeRead read, int indexEntries) { - var books = new Dictionary(); - var stopwatch = Stopwatch.StartNew(); + _ = RunRangeQueries(starts, read); + var elapsed = new double[trials]; + var allocated = new double[trials]; + + for (var trial = 0; trial < trials; trial++) + { + var before = GC.GetAllocatedBytesForCurrentThread(); + var started = Stopwatch.GetTimestamp(); + var sink = RunRangeQueries(starts, read); + var ticks = Stopwatch.GetTimestamp() - started; + elapsed[trial] = ticks * (1_000_000_000.0 / Stopwatch.Frequency) / queries; + allocated[trial] = (GC.GetAllocatedBytesForCurrentThread() - before) / (double)queries; + GC.KeepAlive(sink); + } + + Array.Sort(elapsed); + Array.Sort(allocated); + return new RangeResult(name, queries, indexEntries, Round(elapsed[trials / 2]), + Round(elapsed[0]), Round(elapsed[^1]), Round(allocated[trials / 2])); + } + + private static ulong RunRangeQueries(ulong[] starts, RangeRead read) + { + ulong sink = 0; + + foreach (var from in starts) + { + if (read(from, from + 9, out var found) != JournalRangeResult.Success || + found.Count != 10) + throw new InvalidOperationException("Prepared range was not recoverable."); + sink ^= found[^1].Sequence; + } - var from = Sequencer.None; + return sink; + } + + private static double TimeRecovery(string directory, string checkpointPath) + { + var books = new Dictionary(); + ulong from = Sequencer.None; + var started = Stopwatch.GetTimestamp(); if (checkpointPath is not null) - from = Checkpoint.Restore(checkpointPath, make, books); + from = Checkpoint.Restore(checkpointPath, _ => new SortedArrayBook(16), books, Session); - JournalReader.Recover(directory, (in JournalRecordView record) => + var report = JournalReader.Recover(directory, (in JournalRecordView record) => { if (record.Type != JournalRecordType.Message || record.Sequence <= from) return true; var payload = record.Payload; - var instrument = System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload); + var instrument = BinaryPrimitives.ReadInt32LittleEndian(payload); var side = (Side)payload[4]; - var price = System.Buffers.Binary.BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(5)); - var quantity = System.Buffers.Binary.BinaryPrimitives.ReadUInt32LittleEndian(payload.Slice(9)); + var price = BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(5)); + var quantity = BinaryPrimitives.ReadUInt32LittleEndian(payload.Slice(9)); if (!books.TryGetValue(instrument, out var book)) - books[instrument] = book = make(instrument); - + books[instrument] = book = new SortedArrayBook(16); book.Upsert(side, price, quantity); return true; - }, from); + }, from, Session); + + if (report.Outcome != RecoveryOutcome.Clean) + throw new InvalidDataException("Prepared recovery journal failed validation."); + + GC.KeepAlive(books); + return (Stopwatch.GetTimestamp() - started) * (1_000.0 / Stopwatch.Frequency); + } - stopwatch.Stop(); - return stopwatch.Elapsed.TotalMilliseconds; + private static AppendResult SummariseAppend(string name, int payloadBytes, int records, + double[] elapsed, double[] allocated, long[] syncs, DurabilityPolicy policy, + double intervalMilliseconds) + { + Array.Sort(elapsed); + Array.Sort(allocated); + Array.Sort(syncs); + var median = elapsed[elapsed.Length / 2]; + var rate = 1_000_000_000.0 / median; + + return new AppendResult(name, policy.ToString(), payloadBytes, records, + intervalMilliseconds, syncs[syncs.Length / 2], Round(median), Round(elapsed[0]), + Round(elapsed[^1]), Math.Round(rate, 0), + Math.Round(rate * JournalRecord.SizeFor(payloadBytes) / 1024 / 1024, 2), + Round(allocated[allocated.Length / 2])); } - private static double Median(List values) + private static void Print(AppendResult[] append, RecoveryResult recovery, + RangeResult[] ranges) { - var sorted = values.OrderBy(v => v).ToList(); - return sorted.Count == 0 ? 0 - : sorted.Count % 2 == 1 ? sorted[sorted.Count / 2] - : (sorted[sorted.Count / 2 - 1] + sorted[sorted.Count / 2]) / 2; + Console.WriteLine("Durability acknowledgement path"); + Console.WriteLine($"{"Case",20} {"median ns",12} {"min-max ns",20} {"ops/s",12} {"syncs",8} {"B/op",8}"); + foreach (var item in append) + Console.WriteLine($"{item.Name,20} {item.MedianNanoseconds,12:N1} " + + $"{item.MinNanoseconds:N1}-{item.MaxNanoseconds,10:N1} " + + $"{item.AppendsPerSecond,12:N0} {item.MedianSyncs,8:N0} " + + $"{item.BytesAllocatedPerAppend,8:N3}"); + + Console.WriteLine(); + Console.WriteLine($"Recovery {recovery.Messages:N0}: full {recovery.FullMedianMilliseconds:N2} ms; " + + $"checkpoint@{recovery.CheckpointSequence:N0} {recovery.CheckpointMedianMilliseconds:N2} ms; " + + $"{recovery.SpeedUp:N2}x"); + foreach (var item in ranges) + Console.WriteLine($"Range {item.Name}: {item.MedianNanosecondsPerRequest:N0} ns/request, " + + $"{item.BytesAllocatedPerRequest:N0} B/request, index entries {item.IndexEntries:N0}"); } + + private static double Round(double value) => Math.Round(value, 2); + + private delegate JournalRangeResult RangeRead(ulong from, ulong to, + out List found); + + private sealed record AppendCase(string Name, DurabilityPolicy Policy, TimeSpan Interval, + int GroupSize); + + private sealed record AppendResult(string Name, string Policy, int PayloadBytes, int Records, + double SyncIntervalMilliseconds, long MedianSyncs, double MedianNanoseconds, + double MinNanoseconds, double MaxNanoseconds, double AppendsPerSecond, + double MegabytesPerSecond, double BytesAllocatedPerAppend); + + private sealed record RecoveryResult(int Messages, int CheckpointSequence, + double FullMedianMilliseconds, double FullMinMilliseconds, double FullMaxMilliseconds, + double CheckpointMedianMilliseconds, double CheckpointMinMilliseconds, + double CheckpointMaxMilliseconds, double SpeedUp); + + private sealed record RangeResult(string Name, int Queries, int IndexEntries, + double MedianNanosecondsPerRequest, double MinNanosecondsPerRequest, + double MaxNanosecondsPerRequest, double BytesAllocatedPerRequest); + + private sealed record DurabilityReport(DateTimeOffset TimestampUtc, string Runtime, + string OperatingSystem, string Architecture, int LogicalProcessors, bool ServerGc, + long StopwatchFrequency, string Crc32CImplementation, int Records, int PayloadBytes, + int Trials, AppendResult[] Append, RecoveryResult Recovery, RangeResult[] RangeReads); } } diff --git a/Common/Client/Client.cs b/Common/Client/Client.cs index 4981936..e275c7f 100644 --- a/Common/Client/Client.cs +++ b/Common/Client/Client.cs @@ -17,14 +17,14 @@ public Client(GrpcChannel channel) : base(channel) _streamingTask = StreamAsync(this, _subscriptionChannel.Reader, _shutdownSource); } - private static async Task StreamAsync(Client client, - ChannelReader<(bool Subscribe, int InstrumentId)> reader, + private static async Task StreamAsync(Client client, + ChannelReader<(bool Subscribe, int InstrumentId)> reader, TaskCompletionSource shutdownSource) { using (var streaming = client.StreamOrderbookUpdates()) { _ = HandleResponsesAsync(streaming.ResponseStream, shutdownSource); - + while (true) { diff --git a/Common/Durability/Checkpoint.cs b/Common/Durability/Checkpoint.cs index fd68b16..fe3a0f5 100644 --- a/Common/Durability/Checkpoint.cs +++ b/Common/Durability/Checkpoint.cs @@ -8,98 +8,88 @@ namespace MarketData.Common.Durability { - /// - /// A complete book state captured at one sequence, so recovery need not replay from zero. - /// - /// - /// - /// Without checkpoints, recovery time grows without bound with uptime: a log that has been - /// running for a week takes a week's worth of replay to rebuild. That is the difference between - /// a system that can be restarted during the day and one that cannot. - /// - /// - /// The invariant that makes this safe is precise: a checkpoint at sequence S plus every - /// journalled message after S must reconstruct exactly the same state as replaying every - /// message from the beginning. CheckpointTests asserts that equivalence directly rather - /// than trusting it, because a checkpoint that is subtly wrong is worse than none - it produces - /// a book that is confidently incorrect. - /// - /// - /// Checkpoints are written to their own files rather than inline in the log, and a marker - /// record goes into the log pointing at them. That keeps the log append-only and lets an old - /// checkpoint be deleted without rewriting anything. - /// - /// + /// Versioned, checksummed full-depth state checkpoint. public static class Checkpoint { - public const uint Magic = 0x43484B31; // "CHK1" + public const uint Magic = 0x43484B32; // CHK2 + public const ushort Version = 2; + public const int HeaderSize = 40; + public const int TrailerSize = 8; + public const int MaxCheckpointBytes = 256 * 1024 * 1024; + + private const uint CommitMagic = 0xC04D17ED; private const string Prefix = "checkpoint-"; private const string Suffix = ".chk"; + private const int MaxInstruments = 1_000_000; + private const int CrcOffset = 36; - /// Writes a checkpoint and records a marker in the journal. public static string Write(string directory, WriteAheadJournal journal, ulong sequence, ulong sessionId, IReadOnlyDictionary books) { - Directory.CreateDirectory(directory); + ArgumentException.ThrowIfNullOrWhiteSpace(directory); + ArgumentNullException.ThrowIfNull(journal); + ArgumentNullException.ThrowIfNull(books); + if (sessionId == 0 || sessionId != journal.SessionId) + throw new InvalidDataException("Checkpoint and journal sessions differ."); + if (!journal.HasSequencedRecords || sequence > journal.LastSequence) + throw new InvalidDataException("Checkpoint is outside the durable prefix."); + if (books.Count > MaxInstruments) + throw new ArgumentOutOfRangeException(nameof(books)); + + var instrumentIds = books.Keys.ToArray(); + Array.Sort(instrumentIds); + var payloadLength = PayloadLength(instrumentIds, books); + var totalLength = checked(HeaderSize + payloadLength + TrailerSize); + + if (totalLength > MaxCheckpointBytes) + throw new InvalidDataException("Checkpoint exceeds the configured format bound."); + + var bytes = GC.AllocateUninitializedArray(totalLength); + WriteHeader(bytes, sessionId, sequence, instrumentIds.Length, payloadLength, totalLength); + WritePayload(bytes.AsSpan(HeaderSize, payloadLength), instrumentIds, books); + BinaryPrimitives.WriteInt32LittleEndian(bytes.AsSpan(totalLength - TrailerSize), totalLength); + BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(totalLength - sizeof(uint)), CommitMagic); + var crc = Crc32C.Compute(bytes.AsSpan(0, CrcOffset), + bytes.AsSpan(HeaderSize, payloadLength)); + BinaryPrimitives.WriteUInt32LittleEndian(bytes.AsSpan(CrcOffset), crc); + + Directory.CreateDirectory(directory); var path = Path.Combine(directory, $"{Prefix}{sequence:D20}{Suffix}"); var temporary = path + ".tmp"; - using (var stream = new FileStream(temporary, FileMode.Create, FileAccess.Write, FileShare.None)) - using (var writer = new BinaryWriter(stream)) + try { - writer.Write(Magic); - writer.Write(sessionId); - writer.Write(sequence); - writer.Write(books.Count); - - foreach (var (instrumentId, book) in books.OrderBy(entry => entry.Key)) + using (var stream = new FileStream(temporary, FileMode.Create, FileAccess.Write, + FileShare.None, bufferSize: 1, FileOptions.SequentialScan)) { - writer.Write(instrumentId); - WriteSide(writer, book, Side.Bid); - WriteSide(writer, book, Side.Ask); + stream.Write(bytes); + stream.Flush(flushToDisk: true); } - stream.Flush(flushToDisk: true); - } - - // Rename last, and only once the bytes are on the device. A checkpoint file that - // exists is therefore always complete: a crash mid-write leaves a .tmp that recovery - // ignores, rather than a truncated checkpoint that recovery would trust. - File.Move(temporary, path, overwrite: true); + File.Move(temporary, path, overwrite: true); - Span marker = stackalloc byte[8]; - BinaryPrimitives.WriteUInt64LittleEndian(marker, sequence); - journal.Append(JournalRecordType.Checkpoint, sequence, DateTime.UtcNow.Ticks, marker); - journal.Sync(); - - return path; - } - - private static void WriteSide(BinaryWriter writer, IOrderBook book, Side side) - { - var count = book.Count(side); - var levels = new PriceLevel[count]; - var copied = book.CopyTo(side, levels); - - writer.Write(copied); - - for (var i = 0; i < copied; i++) + Span marker = stackalloc byte[sizeof(ulong)]; + BinaryPrimitives.WriteUInt64LittleEndian(marker, sequence); + journal.Append(JournalRecordType.Checkpoint, sequence, DateTime.UtcNow.Ticks, marker); + journal.Sync(); + return path; + } + finally { - writer.Write(levels[i].Price); - writer.Write(levels[i].Quantity); + if (File.Exists(temporary)) + File.Delete(temporary); } } - /// The newest checkpoint at or below , if any. public static string FindLatest(string directory, ulong notAfter = ulong.MaxValue) { if (!Directory.Exists(directory)) return null; return Directory.GetFiles(directory, Prefix + "*" + Suffix) - .Select(path => (path, sequence: SequenceOf(path))) - .Where(entry => entry.sequence != Sequencer.None && entry.sequence <= notAfter) + .Select(path => (path, valid: TryGetSequence(path, out var sequence), sequence)) + .Where(entry => entry.valid && entry.sequence <= notAfter) .OrderByDescending(entry => entry.sequence) .Select(entry => entry.path) .FirstOrDefault(); @@ -107,75 +97,100 @@ public static string FindLatest(string directory, ulong notAfter = ulong.MaxValu public static ulong SequenceOf(string path) { - var name = Path.GetFileNameWithoutExtension(path); - return name.StartsWith(Prefix, StringComparison.Ordinal) - && ulong.TryParse(name.AsSpan(Prefix.Length), out var sequence) - ? sequence - : Sequencer.None; + return TryGetSequence(path, out var sequence) ? sequence : Sequencer.None; } - /// Restores books from a checkpoint file. - /// The sequence the state is current as of. public static ulong Restore(string path, Func bookFactory, - IDictionary books) + IDictionary books, ulong expectedSessionId = 0) { - using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read); - using var reader = new BinaryReader(stream); + ArgumentException.ThrowIfNullOrWhiteSpace(path); + ArgumentNullException.ThrowIfNull(bookFactory); + ArgumentNullException.ThrowIfNull(books); + + var info = new FileInfo(path); + if (info.Length < HeaderSize + TrailerSize || info.Length > MaxCheckpointBytes) + throw new InvalidDataException("Checkpoint length is invalid."); - if (reader.ReadUInt32() != Magic) - throw new InvalidDataException($"{path} is not a checkpoint."); + var bytes = File.ReadAllBytes(path); + var span = bytes.AsSpan(); - reader.ReadUInt64(); // session id, retained for provenance - var sequence = reader.ReadUInt64(); - var instruments = reader.ReadInt32(); + if (BinaryPrimitives.ReadUInt32LittleEndian(span) != Magic || + BinaryPrimitives.ReadUInt16LittleEndian(span.Slice(4)) != Version || + BinaryPrimitives.ReadUInt16LittleEndian(span.Slice(6)) != HeaderSize) + throw new InvalidDataException("Checkpoint format is unsupported."); - for (var i = 0; i < instruments; i++) + var sessionId = BinaryPrimitives.ReadUInt64LittleEndian(span.Slice(8)); + var sequence = BinaryPrimitives.ReadUInt64LittleEndian(span.Slice(16)); + var instrumentCount = BinaryPrimitives.ReadInt32LittleEndian(span.Slice(24)); + var payloadLength = BinaryPrimitives.ReadInt32LittleEndian(span.Slice(28)); + var totalLength = BinaryPrimitives.ReadInt32LittleEndian(span.Slice(32)); + + if (sessionId == 0 || (expectedSessionId != 0 && sessionId != expectedSessionId)) + throw new InvalidDataException("Checkpoint session does not match."); + if (instrumentCount < 0 || instrumentCount > MaxInstruments || payloadLength < 0 || + totalLength != bytes.Length || totalLength != HeaderSize + payloadLength + TrailerSize) + throw new InvalidDataException("Checkpoint framing is invalid."); + if (BinaryPrimitives.ReadInt32LittleEndian(span.Slice(totalLength - TrailerSize)) != + totalLength || + BinaryPrimitives.ReadUInt32LittleEndian(span.Slice(totalLength - sizeof(uint))) != + CommitMagic) + throw new InvalidDataException("Checkpoint commit trailer is invalid."); + + var storedCrc = BinaryPrimitives.ReadUInt32LittleEndian(span.Slice(CrcOffset)); + var actualCrc = Crc32C.Compute(span.Slice(0, CrcOffset), + span.Slice(HeaderSize, payloadLength)); + if (storedCrc != actualCrc) + throw new InvalidDataException("Checkpoint checksum failed."); + + var restored = new Dictionary(instrumentCount); + var payload = span.Slice(HeaderSize, payloadLength); + var offset = 0; + + for (var i = 0; i < instrumentCount; i++) { - var instrumentId = reader.ReadInt32(); + EnsureRemaining(payload, offset, 12); + var instrumentId = BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(offset)); + var bidCount = BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(offset + 4)); + var askCount = BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(offset + 8)); + offset += 12; + + var maxLevels = (payload.Length - offset) / 8; + if (instrumentId <= 0 || bidCount < 0 || askCount < 0 || + bidCount > maxLevels || askCount > maxLevels || + restored.ContainsKey(instrumentId)) + throw new InvalidDataException("Checkpoint instrument table is invalid."); + var book = bookFactory(instrumentId); - book.Clear(); + if (book is null || !restored.TryAdd(instrumentId, book)) + throw new InvalidDataException("Checkpoint book factory failed."); - ReadSide(reader, book, Side.Bid); - ReadSide(reader, book, Side.Ask); + if (bidCount > book.Depth || askCount > book.Depth) + throw new InvalidDataException("Checkpoint depth exceeds the target book."); - books[instrumentId] = book; + offset = ReadSide(payload, offset, bidCount, Side.Bid, book); + offset = ReadSide(payload, offset, askCount, Side.Ask, book); } - return sequence; - } + if (offset != payload.Length) + throw new InvalidDataException("Checkpoint payload has trailing data."); - private static void ReadSide(BinaryReader reader, IOrderBook book, Side side) - { - var count = reader.ReadInt32(); + books.Clear(); + foreach (var entry in restored) + books.Add(entry.Key, entry.Value); - for (var i = 0; i < count; i++) - { - var price = reader.ReadInt32(); - var quantity = reader.ReadUInt32(); - book.Upsert(side, price, quantity); - } + return sequence; } - /// - /// Deletes checkpoints older than the newest . - /// - /// - /// Never deletes the newest, whatever says: a directory with no - /// checkpoint at all is exactly the unbounded-recovery situation checkpoints exist to - /// prevent, and retention should not be able to cause it. - /// public static int Prune(string directory, int keep = 3) { if (keep < 1) - throw new ArgumentOutOfRangeException(nameof(keep), keep, "Must keep at least one."); - + throw new ArgumentOutOfRangeException(nameof(keep)); if (!Directory.Exists(directory)) return 0; var ordered = Directory.GetFiles(directory, Prefix + "*" + Suffix) .OrderByDescending(SequenceOf) .ToList(); - var removed = 0; foreach (var path in ordered.Skip(keep)) @@ -186,5 +201,112 @@ public static int Prune(string directory, int keep = 3) return removed; } + + private static int PayloadLength(int[] instrumentIds, + IReadOnlyDictionary books) + { + var length = 0; + + foreach (var instrumentId in instrumentIds) + { + var book = books[instrumentId] ?? + throw new InvalidDataException("Checkpoint book cannot be null."); + length = checked(length + 12 + checked((book.Count(Side.Bid) + + book.Count(Side.Ask)) * 8)); + } + + return length; + } + + private static bool TryGetSequence(string path, out ulong sequence) + { + var name = Path.GetFileNameWithoutExtension(path); + sequence = Sequencer.None; + return name.StartsWith(Prefix, StringComparison.Ordinal) && + ulong.TryParse(name.AsSpan(Prefix.Length), out sequence); + } + + private static void WriteHeader(Span destination, ulong sessionId, ulong sequence, + int instruments, int payloadLength, int totalLength) + { + BinaryPrimitives.WriteUInt32LittleEndian(destination, Magic); + BinaryPrimitives.WriteUInt16LittleEndian(destination.Slice(4), Version); + BinaryPrimitives.WriteUInt16LittleEndian(destination.Slice(6), HeaderSize); + BinaryPrimitives.WriteUInt64LittleEndian(destination.Slice(8), sessionId); + BinaryPrimitives.WriteUInt64LittleEndian(destination.Slice(16), sequence); + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(24), instruments); + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(28), payloadLength); + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(32), totalLength); + BinaryPrimitives.WriteUInt32LittleEndian(destination.Slice(CrcOffset), 0); + } + + private static void WritePayload(Span destination, int[] instrumentIds, + IReadOnlyDictionary books) + { + var offset = 0; + + foreach (var instrumentId in instrumentIds) + { + var book = books[instrumentId]; + var bids = new PriceLevel[book.Count(Side.Bid)]; + var asks = new PriceLevel[book.Count(Side.Ask)]; + var bidCount = book.CopyTo(Side.Bid, bids); + var askCount = book.CopyTo(Side.Ask, asks); + + if (bidCount != bids.Length || askCount != asks.Length) + throw new InvalidOperationException("Book changed while checkpointing."); + + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset), instrumentId); + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset + 4), bidCount); + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset + 8), askCount); + offset += 12; + offset = WriteLevels(destination, offset, bids); + offset = WriteLevels(destination, offset, asks); + } + } + + private static int WriteLevels(Span destination, int offset, PriceLevel[] levels) + { + foreach (var level in levels) + { + if (level.Quantity == 0) + throw new InvalidDataException("Checkpoint contains an empty level."); + + BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(offset), level.Price); + BinaryPrimitives.WriteUInt32LittleEndian(destination.Slice(offset + 4), level.Quantity); + offset += 8; + } + + return offset; + } + + private static int ReadSide(ReadOnlySpan payload, int offset, int count, Side side, + IOrderBook book) + { + EnsureRemaining(payload, offset, checked(count * 8)); + var previous = 0; + + for (var i = 0; i < count; i++) + { + var price = BinaryPrimitives.ReadInt32LittleEndian(payload.Slice(offset)); + var quantity = BinaryPrimitives.ReadUInt32LittleEndian(payload.Slice(offset + 4)); + + if (quantity == 0 || (i > 0 && (side == Side.Bid ? price >= previous : price <= previous))) + throw new InvalidDataException("Checkpoint levels are not canonical."); + if (!book.Upsert(side, price, quantity)) + throw new InvalidDataException("Checkpoint level could not be restored."); + + previous = price; + offset += 8; + } + + return offset; + } + + private static void EnsureRemaining(ReadOnlySpan payload, int offset, int required) + { + if (offset < 0 || required < 0 || offset > payload.Length - required) + throw new InvalidDataException("Checkpoint payload is truncated."); + } } } diff --git a/Common/Durability/JournalRangeReader.cs b/Common/Durability/JournalRangeReader.cs new file mode 100644 index 0000000..50caf60 --- /dev/null +++ b/Common/Durability/JournalRangeReader.cs @@ -0,0 +1,299 @@ +using System; +using System.Buffers; +using System.Collections.Generic; +using System.IO; +using MarketData.Common.Feed; + +namespace MarketData.Common.Durability +{ + /// Sparse in-memory sequence index with incremental tail refresh. + public sealed class JournalRangeReader + { + public const int DefaultStride = 256; + + private readonly object _gate = new(); + private readonly string _directory; + private readonly int _stride; + private readonly List _entries = new(); + + private List _segments = new(); + private int _scanSegment = -1; + private long _scanOffset; + private long _sequencedRecords; + private ulong _nextSequence; + + public JournalRangeReader(string directory, int stride = DefaultStride) + { + ArgumentException.ThrowIfNullOrWhiteSpace(directory); + if (stride is < 1 or > 65_536) + throw new ArgumentOutOfRangeException(nameof(stride)); + + _directory = directory; + _stride = stride; + Build(); + } + + public ulong SessionId { get; private set; } + public int IndexEntries { get { lock (_gate) return _entries.Count; } } + + public JournalRangeResult TryRead(ulong sessionId, ulong from, ulong to, + out List messages) + { + if (to < from) + throw new ArgumentOutOfRangeException(nameof(to)); + + IReadOnlyList segments; + IndexEntry entry; + ulong indexedSession; + + lock (_gate) + { + if (to >= _nextSequence) + Refresh(); + + if (sessionId != 0 && sessionId != SessionId) + { + messages = new List(); + return JournalRangeResult.WrongSession; + } + + var index = FindFloor(from); + if (index < 0) + return JournalReader.TryReadRange(_directory, SessionId, from, to, out messages); + + entry = _entries[index]; + segments = _segments; + indexedSession = SessionId; + } + + var result = JournalReader.TryReadRangeFrom(segments, entry.Segment, entry.Offset, + indexedSession, entry.Sequence, from, to, out messages); + + if (result != JournalRangeResult.Success) + messages.Clear(); + + return result; + } + + private void Build() + { + _entries.Clear(); + _segments = WriteAheadJournal.SegmentFiles(_directory); + _scanSegment = -1; + _scanOffset = 0; + _sequencedRecords = 0; + + var report = JournalReader.Recover(_directory, (in JournalRecordView record) => + { + if (record.Type == JournalRecordType.SegmentHeader) + { + _scanSegment++; + _scanOffset = record.TotalSize; + return true; + } + + if (TryGetFirstSequence(record, out var sequence)) + { + if (_sequencedRecords % _stride == 0) + _entries.Add(new IndexEntry(sequence, _scanSegment, _scanOffset)); + _sequencedRecords++; + } + + _scanOffset += record.TotalSize; + return true; + }); + + if (report.Outcome == RecoveryOutcome.Corrupt || report.SessionId == 0) + throw new InvalidDataException("Cannot index an invalid journal."); + + SessionId = report.SessionId; + _nextSequence = report.NextSequence; + } + + private void Refresh() + { + var current = WriteAheadJournal.SegmentFiles(_directory); + + if (!HasStablePrefix(current)) + { + Build(); + return; + } + + _segments = current; + + while (true) + { + if (_scanSegment < 0) + { + if (!TryOpenNextSegment()) + return; + } + + using (var stream = new FileStream(_segments[_scanSegment], FileMode.Open, + FileAccess.Read, FileShare.ReadWrite | FileShare.Delete, bufferSize: 1, + FileOptions.SequentialScan)) + { + var limit = stream.Length; + if (_scanOffset > limit) + throw new InvalidDataException("Journal segment shrank under the index."); + stream.Position = _scanOffset; + var reader = new JournalReader.PooledReader(stream, limit); + + try + { + while (reader.Position < limit) + { + var offset = reader.Position; + var result = JournalReader.ReadRecord(ref reader, out var rented, + out var size); + + if (result == JournalReadResult.Incomplete) + return; + if (result != JournalReadResult.Ok) + throw new InvalidDataException( + $"Journal tail failed validation: {result}."); + + try + { + JournalRecord.TryRead(rented.AsSpan(0, size), out var record); + if (!Advance(record, offset)) + throw new InvalidDataException( + "Journal tail broke sequence continuity."); + _scanOffset = reader.Position; + } + finally + { + ArrayPool.Shared.Return(rented); + } + } + } + finally + { + reader.Dispose(); + } + } + + if (_scanSegment + 1 >= _segments.Count || !TryOpenNextSegment()) + return; + } + } + + private bool TryOpenNextSegment() + { + var next = _scanSegment + 1; + if (next >= _segments.Count) + return false; + + using var stream = new FileStream(_segments[next], FileMode.Open, FileAccess.Read, + FileShare.ReadWrite | FileShare.Delete, bufferSize: 1, + FileOptions.SequentialScan); + var result = JournalReader.ReadRecord(stream, stream.Length, out var rented, out var size); + + if (result == JournalReadResult.Incomplete) + return false; + if (result != JournalReadResult.Ok) + throw new InvalidDataException($"Journal segment header failed validation: {result}."); + + try + { + JournalRecord.TryRead(rented.AsSpan(0, size), out var record); + if (record.Type != JournalRecordType.SegmentHeader || record.Payload.Length != 16 || + System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian(record.Payload) != + SessionId || + System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian( + record.Payload.Slice(8)) != _nextSequence) + throw new InvalidDataException("Journal segment header is discontinuous."); + + if (_scanSegment >= 0 && WriteAheadJournal.IndexOf(_segments[next]) != + WriteAheadJournal.IndexOf(_segments[_scanSegment]) + 1) + throw new InvalidDataException("Journal segment index is discontinuous."); + + _scanSegment = next; + _scanOffset = size; + return true; + } + finally + { + ArrayPool.Shared.Return(rented); + } + } + + private bool Advance(in JournalRecordView record, long offset) + { + if (record.Type is JournalRecordType.Checkpoint or JournalRecordType.Audit) + return record.Sequence == Sequencer.None || record.Sequence < _nextSequence; + if (record.Type == JournalRecordType.FeedPacket && + (!FeedProtocol.TryReadHeader(record.Payload, out var header, out _) || + header.SessionId != SessionId)) + return false; + if (!TryGetRange(record, out var first, out var last) || first != _nextSequence) + return false; + + if (_sequencedRecords % _stride == 0) + _entries.Add(new IndexEntry(first, _scanSegment, offset)); + _sequencedRecords++; + _nextSequence = last == ulong.MaxValue ? ulong.MaxValue : last + 1; + return true; + } + + private bool HasStablePrefix(List current) + { + if (current.Count < _segments.Count) + return false; + + for (var i = 0; i < _segments.Count; i++) + if (!string.Equals(current[i], _segments[i], StringComparison.Ordinal)) + return false; + + return true; + } + + private int FindFloor(ulong sequence) + { + var low = 0; + var high = _entries.Count - 1; + var found = -1; + + while (low <= high) + { + var middle = low + ((high - low) >> 1); + if (_entries[middle].Sequence <= sequence) + { + found = middle; + low = middle + 1; + } + else + { + high = middle - 1; + } + } + + return found; + } + + private static bool TryGetFirstSequence(in JournalRecordView record, out ulong sequence) + { + sequence = record.Sequence; + return record.Type is JournalRecordType.Message or JournalRecordType.FeedPacket; + } + + private static bool TryGetRange(in JournalRecordView record, out ulong first, out ulong last) + { + first = record.Sequence; + last = record.Sequence; + + if (record.Type == JournalRecordType.Message) + return first != ulong.MaxValue; + if (record.Type != JournalRecordType.FeedPacket || + !FeedProtocol.TryReadHeader(record.Payload, out var header, out _)) + return false; + + first = header.FirstSequence; + last = first + header.MessageCount - 1; + return true; + } + + private readonly record struct IndexEntry(ulong Sequence, int Segment, long Offset); + } +} diff --git a/Common/Durability/JournalReader.cs b/Common/Durability/JournalReader.cs index 5f5f0dd..95af760 100644 --- a/Common/Durability/JournalReader.cs +++ b/Common/Durability/JournalReader.cs @@ -1,26 +1,16 @@ using System; using System.Buffers; +using System.Buffers.Binary; using System.Collections.Generic; using System.IO; +using MarketData.Common.Feed; namespace MarketData.Common.Durability { - /// What recovery found at the end of the log. public enum RecoveryOutcome { - /// Every record in every segment validated. Clean, - - /// - /// The final record was incomplete. Expected after a crash mid-append, and recoverable: - /// everything before it is intact and the partial tail is discarded. - /// TruncatedTail, - - /// - /// A record failed validation with more data after it. This is not a torn tail - it is - /// damage inside the log - and it bounds what can be trusted. - /// Corrupt, } @@ -31,192 +21,723 @@ public sealed record RecoveryReport( ulong LastCheckpointSequence, long ValidBytes, string DamagedSegment, - JournalReadResult Failure) + JournalReadResult Failure, + ulong SessionId = 0, + ulong NextSequence = 0, + bool HasSequencedRecords = false, + long ValidBytesInDamagedSegment = 0) { - /// Whether the log can be resumed without operator intervention. public bool Resumable => Outcome != RecoveryOutcome.Corrupt; } - /// Reads a journal back, for recovery, catch-up and retransmission. + public enum JournalRangeResult : byte + { + Success, + Missing, + WrongSession, + Corrupt, + } + + /// Streaming, allocation-bounded journal validation and replay. public static class JournalReader { - /// - /// Scans every segment, validating as it goes. - /// - /// - /// Invoked for each valid record in sequence order. Return false to stop early. - /// - /// - /// - /// The distinction this method exists to draw is between a torn tail and - /// corruption. A process killed mid-append leaves a partial record at the very end - /// of the last segment; that is normal, expected, and safe to discard, because nothing was - /// ever acknowledged from it. A record that fails to validate with more data after it means - /// something overwrote the middle of the log, and silently skipping it would hand callers a - /// log with an invisible hole. - /// - /// - /// So the two are reported differently and only the first is recoverable. Anything else is - /// a decision for an operator, not for a library. - /// - /// + internal const int ReadBufferSize = 64 * 1024; + private const int RangeReadBufferSize = 4 * 1024; + public static RecoveryReport Recover(string directory, RecordHandler onRecord = null, - ulong fromSequence = Sequencer.None) + ulong fromSequence = Sequencer.None, ulong expectedSessionId = 0, + ulong? expectedInitialSequence = null) { - var segments = WriteAheadJournal.SegmentFiles(directory); + var paths = WriteAheadJournal.SegmentFiles(directory); - // Skip whole segments that end before the caller's starting point. Without this, - // restoring from a checkpoint still reads and checksums the entire history behind it, - // so recovery stays O(uptime) and the checkpoint buys almost nothing - which is what - // the first version of this actually did, and the benchmark caught it. - var firstSegment = 0; + if (paths.Count == 0) + return Clean(0, Sequencer.None, Sequencer.None, 0, 0, + expectedInitialSequence ?? Sequencer.None, false); - if (fromSequence != Sequencer.None) + var segments = new List(paths.Count); + var priorIndex = -1; + ulong sessionId = 0; + string incompleteTail = null; + + for (var i = 0; i < paths.Count; i++) { - for (var s = 0; s + 1 < segments.Count; s++) - { - // A segment can be skipped only once the *next* segment is known to start at - // or before the target: that proves this one holds nothing the caller wants. - var nextStart = SegmentFirstSequence(segments[s + 1]); + var path = paths[i]; + var result = ReadSegmentHeader(path, out var descriptor); - if (nextStart != Sequencer.None && nextStart <= fromSequence) - firstSegment = s + 1; - else + if (result != JournalReadResult.Ok) + { + if (result == JournalReadResult.Incomplete && i == paths.Count - 1) + { + incompleteTail = path; break; + } + + return Corrupt(path, result, sessionId); } + + if ((priorIndex < 0 && descriptor.Index != 0) || + (priorIndex >= 0 && descriptor.Index != priorIndex + 1)) + return Corrupt(path, JournalReadResult.SegmentOrder, sessionId); + + if (sessionId == 0) + sessionId = descriptor.SessionId; + else if (descriptor.SessionId != sessionId) + return Corrupt(path, JournalReadResult.BadSession, sessionId); + + if (expectedSessionId != 0 && descriptor.SessionId != expectedSessionId) + return Corrupt(path, JournalReadResult.BadSession, descriptor.SessionId); + + priorIndex = descriptor.Index; + segments.Add(descriptor); } - segments = segments.GetRange(firstSegment, segments.Count - firstSegment); + if (segments.Count > 0 && expectedInitialSequence.HasValue && + segments[0].FirstSequence != expectedInitialSequence.Value) + return Corrupt(segments[0].Path, JournalReadResult.SequenceGap, sessionId); + var firstSegment = SelectFirstSegment(segments, fromSequence); long records = 0; long validBytes = 0; ulong lastSequence = Sequencer.None; ulong lastCheckpoint = Sequencer.None; + var hasSequencedRecords = false; + var expected = firstSegment < segments.Count + ? segments[firstSegment].FirstSequence + : expectedInitialSequence ?? Sequencer.None; - for (var s = 0; s < segments.Count; s++) + for (var i = firstSegment; i < segments.Count; i++) { - var path = segments[s]; - var isLastSegment = s == segments.Count - 1; - var bytes = File.ReadAllBytes(path); - var offset = 0; + var segment = segments[i]; - while (offset < bytes.Length) + if (i > firstSegment && segment.FirstSequence != expected) { - var result = JournalRecord.TryRead(bytes.AsSpan(offset), out var record); + return new RecoveryReport(RecoveryOutcome.Corrupt, records, lastSequence, + lastCheckpoint, validBytes, segment.Path, JournalReadResult.SequenceGap, + sessionId, expected, hasSequencedRecords, 0); + } - if (result == JournalReadResult.Ok) - { - records++; - validBytes += record.TotalSize; + var report = ScanSegment(segment, i == paths.Count - 1 && incompleteTail is null, + sessionId, ref expected, ref hasSequencedRecords, ref lastSequence, + ref lastCheckpoint, ref records, ref validBytes, onRecord); - if (record.Type == JournalRecordType.Checkpoint) - lastCheckpoint = record.Sequence; + if (report is not null) + return report; + } - if (record.Type != JournalRecordType.SegmentHeader && record.Sequence > lastSequence) - lastSequence = record.Sequence; + if (incompleteTail is not null) + { + return new RecoveryReport(RecoveryOutcome.TruncatedTail, records, lastSequence, + lastCheckpoint, validBytes, incompleteTail, JournalReadResult.Incomplete, + sessionId, expected, hasSequencedRecords, 0); + } - if (onRecord is not null && !onRecord(record)) - return new RecoveryReport(RecoveryOutcome.Clean, records, lastSequence, - lastCheckpoint, validBytes, null, JournalReadResult.Ok); + return Clean(records, lastSequence, lastCheckpoint, validBytes, sessionId, + expected, hasSequencedRecords); + } - offset += record.TotalSize; - continue; + public static List ReadRange(string directory, ulong from, ulong to) + { + var result = TryReadRange(directory, 0, from, to, out var found); + + if (result == JournalRangeResult.Corrupt) + throw new InvalidDataException("The journal failed validation."); + + return found; + } + + public static JournalRangeResult TryReadRange(string directory, ulong sessionId, ulong from, + ulong to, out List found) + { + if (to < from) + throw new ArgumentOutOfRangeException(nameof(to)); + + var results = new List(RangeCapacity(from, to)); + found = results; + var cursor = from; + var complete = false; + + var report = Recover(directory, (in JournalRecordView record) => + { + if (!TryGetSequenceRange(record, out var first, out var last, out var count)) + return true; + + if (last < from) + return true; + if (first > to || first > cursor) + return false; + + if (first != cursor || last > to) + return false; + + if (last >= cursor) + { + results.Add(new SequencedPayload(first, record.Timestamp, record.Payload.ToArray()) + { + MessageCount = count, + }); + + if (last == ulong.MaxValue || last >= to) + { + complete = true; + return false; } - // A short read at the very end of the very last segment is a torn tail. - // Anywhere else, the log has a hole in the middle and callers must be told. - var atEndOfLastSegment = isLastSegment; + cursor = last + 1; + } + + return true; + }, from, sessionId); + + if (report.SessionId != 0) + { + for (var i = 0; i < results.Count; i++) + results[i] = results[i] with { SessionId = report.SessionId }; + } + + if (report.Failure == JournalReadResult.BadSession) + { + results.Clear(); + return JournalRangeResult.WrongSession; + } + if (report.Outcome == RecoveryOutcome.Corrupt) + { + results.Clear(); + return JournalRangeResult.Corrupt; + } - var outcome = result == JournalReadResult.Incomplete && atEndOfLastSegment - ? RecoveryOutcome.TruncatedTail - : RecoveryOutcome.Corrupt; + if (complete) + return JournalRangeResult.Success; - return new RecoveryReport(outcome, records, lastSequence, lastCheckpoint, - validBytes, path, result); + results.Clear(); + return JournalRangeResult.Missing; + } + + internal static JournalRangeResult TryReadRangeFrom(IReadOnlyList segments, + int firstSegment, long firstOffset, ulong sessionId, ulong journalExpected, + ulong from, ulong to, out List found) + { + found = new List(RangeCapacity(from, to)); + var cursor = from; + var expected = journalExpected; + + for (var segmentIndex = firstSegment; segmentIndex < segments.Count; segmentIndex++) + { + using var stream = new FileStream(segments[segmentIndex], FileMode.Open, + FileAccess.Read, FileShare.ReadWrite | FileShare.Delete, bufferSize: 1, + FileOptions.SequentialScan); + var limit = stream.Length; + + if (segmentIndex == firstSegment) + { + if (firstOffset < 0 || firstOffset > limit) + return JournalRangeResult.Corrupt; + stream.Position = firstOffset; + } + + var reader = new PooledReader(stream, limit, RangeReadBufferSize); + + try + { + if (segmentIndex != firstSegment) + { + var headerResult = ReadRecord(ref reader, out var headerBuffer, + out var headerSize); + if (headerResult != JournalReadResult.Ok) + return headerResult == JournalReadResult.Incomplete + ? JournalRangeResult.Missing + : JournalRangeResult.Corrupt; + + try + { + JournalRecord.TryRead(headerBuffer.AsSpan(0, headerSize), + out var headerRecord); + if (headerRecord.Type != JournalRecordType.SegmentHeader || + headerRecord.Payload.Length != 16 || + BinaryPrimitives.ReadUInt64LittleEndian(headerRecord.Payload) != + sessionId || + BinaryPrimitives.ReadUInt64LittleEndian( + headerRecord.Payload.Slice(8)) != expected) + return JournalRangeResult.Corrupt; + } + finally + { + ArrayPool.Shared.Return(headerBuffer); + } + } + + while (reader.Position < limit) + { + var result = ReadRecord(ref reader, out var rented, out var size); + if (result != JournalReadResult.Ok) + return result == JournalReadResult.Incomplete + ? JournalRangeResult.Missing + : JournalRangeResult.Corrupt; + + try + { + JournalRecord.TryRead(rented.AsSpan(0, size), out var record); + + if (record.Type is JournalRecordType.Checkpoint or JournalRecordType.Audit) + continue; + if (!TryGetSequenceRange(record, out var first, out var last, + out var count) || first != expected) + return JournalRangeResult.Corrupt; + + if (record.Type == JournalRecordType.FeedPacket && + (!FeedProtocol.TryReadHeader(record.Payload, out var feedHeader, + out _) || feedHeader.SessionId != sessionId)) + return JournalRangeResult.Corrupt; + + expected = last == ulong.MaxValue ? ulong.MaxValue : last + 1; + + if (last < from) + continue; + if (first > to || first > cursor || first != cursor || last > to) + return JournalRangeResult.Missing; + + found.Add(new SequencedPayload(first, record.Timestamp, + record.Payload.ToArray()) + { + SessionId = sessionId, + MessageCount = count, + }); + + if (last == ulong.MaxValue || last >= to) + return JournalRangeResult.Success; + + cursor = last + 1; + } + finally + { + ArrayPool.Shared.Return(rented); + } + } + } + finally + { + reader.Dispose(); } } - return new RecoveryReport(RecoveryOutcome.Clean, records, lastSequence, lastCheckpoint, - validBytes, null, JournalReadResult.Ok); + return JournalRangeResult.Missing; } - /// - /// Reads the messages in [from, to], which is what a gap-fill request asks for. - /// - /// - /// Returns only records: a subscriber filling a gap - /// wants the market data it missed, not the journal's own bookkeeping. - /// - public static List ReadRange(string directory, ulong from, ulong to) + private static RecoveryReport ScanSegment(SegmentDescriptor segment, bool isLastSegment, + ulong sessionId, ref ulong expected, ref bool hasSequencedRecords, ref ulong lastSequence, + ref ulong lastCheckpoint, ref long records, ref long validBytes, RecordHandler onRecord) { - if (to < from) - throw new ArgumentOutOfRangeException(nameof(to), "Range ends before it starts."); + using var stream = new FileStream(segment.Path, FileMode.Open, FileAccess.Read, + FileShare.ReadWrite | FileShare.Delete, bufferSize: 1, FileOptions.SequentialScan); - var found = new List(); + var limit = stream.Length; + long offset = 0; + var firstRecord = true; + var reader = new PooledReader(stream, limit); - Recover(directory, (in JournalRecordView record) => + try { - if (record.Type == JournalRecordType.Message && - record.Sequence >= from && record.Sequence <= to) + while (offset < limit) { - found.Add(new SequencedPayload(record.Sequence, record.Timestamp, - record.Payload.ToArray())); + var result = ReadRecord(ref reader, out var rented, out var size); + + if (result != JournalReadResult.Ok) + { + var outcome = result == JournalReadResult.Incomplete && isLastSegment + ? RecoveryOutcome.TruncatedTail + : RecoveryOutcome.Corrupt; + + return new RecoveryReport(outcome, records, lastSequence, lastCheckpoint, + validBytes, segment.Path, result, sessionId, expected, + hasSequencedRecords, offset); + } + + try + { + JournalRecord.TryRead(rented.AsSpan(0, size), out var record); + + if (firstRecord) + { + firstRecord = false; + if (!IsExpectedHeader(record, segment)) + { + return new RecoveryReport(RecoveryOutcome.Corrupt, records, + lastSequence, lastCheckpoint, validBytes, segment.Path, + JournalReadResult.SegmentOrder, sessionId, expected, + hasSequencedRecords, offset); + } + } + else if (!ApplySequenceContract(record, sessionId, ref expected, + ref hasSequencedRecords, ref lastSequence, ref lastCheckpoint, + out var semanticFailure)) + { + return new RecoveryReport(RecoveryOutcome.Corrupt, records, lastSequence, + lastCheckpoint, validBytes, segment.Path, semanticFailure, + sessionId, expected, hasSequencedRecords, offset); + } + + records++; + validBytes += size; + offset += size; + + if (onRecord is not null && !onRecord(record)) + return Clean(records, lastSequence, lastCheckpoint, validBytes, sessionId, + expected, hasSequencedRecords); + } + finally + { + ArrayPool.Shared.Return(rented); + } } + } + finally + { + reader.Dispose(); + } - // Sequences ascend, so once past the range there is nothing left to find. - return record.Sequence <= to; - }); + return null; + } - return found; + private static bool ApplySequenceContract(in JournalRecordView record, ulong sessionId, + ref ulong expected, ref bool hasSequencedRecords, ref ulong lastSequence, + ref ulong lastCheckpoint, out JournalReadResult failure) + { + failure = JournalReadResult.Ok; + + switch (record.Type) + { + case JournalRecordType.Message: + if (record.Sequence != expected || expected == ulong.MaxValue) + { + failure = JournalReadResult.SequenceGap; + return false; + } + + lastSequence = record.Sequence; + hasSequencedRecords = true; + expected++; + return true; + + case JournalRecordType.FeedPacket: + if (!FeedProtocol.TryReadHeader(record.Payload, out var header, out _) || + header.SessionId != sessionId || header.FirstSequence != record.Sequence || + header.FirstSequence != expected) + { + failure = header.SessionId != 0 && header.SessionId != sessionId + ? JournalReadResult.BadSession + : JournalReadResult.SequenceGap; + return false; + } + + expected = header.FirstSequence + header.MessageCount; + lastSequence = expected - 1; + hasSequencedRecords = true; + return true; + + case JournalRecordType.Checkpoint: + if (record.Payload.Length != sizeof(ulong) || + BinaryPrimitives.ReadUInt64LittleEndian(record.Payload) != record.Sequence || + (hasSequencedRecords ? record.Sequence > lastSequence : + record.Sequence != Sequencer.None)) + { + failure = JournalReadResult.SequenceGap; + return false; + } + + lastCheckpoint = record.Sequence; + return true; + + case JournalRecordType.Audit: + if (hasSequencedRecords ? record.Sequence > lastSequence : + record.Sequence != Sequencer.None) + { + failure = JournalReadResult.SequenceGap; + return false; + } + + return true; + + default: + failure = JournalReadResult.BadType; + return false; + } } - /// - /// Reads the sequence a segment starts at, from its own header record. - /// - /// - /// Read from the file's contents rather than parsed out of its name. The name is a - /// convenience for humans and for ordering; trusting it for correctness would make a - /// renamed or copied file silently wrong. - /// - private static ulong SegmentFirstSequence(string path) + private static bool TryGetSequenceRange(in JournalRecordView record, out ulong first, + out ulong last, out ushort count) { - try + first = record.Sequence; + last = record.Sequence; + count = 1; + + if (record.Type == JournalRecordType.Message) + return true; + if (record.Type != JournalRecordType.FeedPacket || + !FeedProtocol.TryReadHeader(record.Payload, out var header, out _)) + return false; + + first = header.FirstSequence; + count = header.MessageCount; + last = first + count - 1; + return true; + } + + private static int RangeCapacity(ulong from, ulong to) + { + var distance = to - from; + return distance >= 255 ? 256 : (int)distance + 1; + } + + internal static JournalReadResult ReadRecord(ref PooledReader reader, + out byte[] rented, out int size) + { + rented = null; + size = 0; + Span header = stackalloc byte[JournalRecord.HeaderSize]; + + if (!reader.ReadExactly(header)) + return JournalReadResult.Incomplete; + + var headerResult = JournalRecord.TryGetSize(header, out size); + if (headerResult != JournalReadResult.Ok) + return headerResult; + + rented = ArrayPool.Shared.Rent(size); + header.CopyTo(rented); + + if (!reader.ReadExactly(rented.AsSpan(JournalRecord.HeaderSize, + size - JournalRecord.HeaderSize))) + { + ArrayPool.Shared.Return(rented); + rented = null; + return JournalReadResult.Incomplete; + } + + var result = JournalRecord.TryRead(rented.AsSpan(0, size), out _); + if (result != JournalReadResult.Ok) + { + ArrayPool.Shared.Return(rented); + rented = null; + } + + return result; + } + + internal static JournalReadResult ReadRecord(FileStream stream, long limit, + out byte[] rented, out int size) + { + rented = null; + size = 0; + Span header = stackalloc byte[JournalRecord.HeaderSize]; + + if (!ReadExactly(stream, header, limit)) + return JournalReadResult.Incomplete; + + var headerResult = JournalRecord.TryGetSize(header, out size); + if (headerResult != JournalReadResult.Ok) + return headerResult; + + rented = ArrayPool.Shared.Rent(size); + header.CopyTo(rented); + + if (!ReadExactly(stream, rented.AsSpan(JournalRecord.HeaderSize, + size - JournalRecord.HeaderSize), limit)) + { + ArrayPool.Shared.Return(rented); + rented = null; + return JournalReadResult.Incomplete; + } + + var result = JournalRecord.TryRead(rented.AsSpan(0, size), out _); + if (result != JournalReadResult.Ok) + { + ArrayPool.Shared.Return(rented); + rented = null; + } + + return result; + } + + private static bool ReadExactly(FileStream stream, Span destination, long limit) + { + var read = 0; + + while (read < destination.Length && stream.Position < limit) + { + var available = (int)Math.Min(destination.Length - read, limit - stream.Position); + var got = stream.Read(destination.Slice(read, available)); + if (got == 0) + break; + read += got; + } + + return read == destination.Length; + } + + internal struct PooledReader : IDisposable + { + private readonly FileStream _stream; + private readonly long _limit; + private byte[] _buffer; + private int _offset; + private int _count; + private long _position; + + public PooledReader(FileStream stream, long limit, int bufferSize = ReadBufferSize) { - using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); + _stream = stream; + _limit = limit; + _buffer = ArrayPool.Shared.Rent(bufferSize); + _offset = 0; + _count = 0; + _position = stream.Position; + } - var header = new byte[JournalRecord.SizeFor(16)]; - var read = 0; + public readonly long Position => _position; - while (read < header.Length) + public bool ReadExactly(Span destination) + { + var written = 0; + + while (written < destination.Length) { - var got = stream.Read(header, read, header.Length - read); - if (got == 0) break; - read += got; + if (_offset == _count && !Fill()) + return false; + + var available = Math.Min(destination.Length - written, _count - _offset); + _buffer.AsSpan(_offset, available).CopyTo(destination.Slice(written)); + _offset += available; + _position += available; + written += available; } - if (read < header.Length) - return Sequencer.None; + return true; + } - if (JournalRecord.TryRead(header, out var record) != JournalReadResult.Ok || - record.Type != JournalRecordType.SegmentHeader || - record.Payload.Length < 16) + private bool Fill() + { + _offset = 0; + _count = 0; + + if (_position >= _limit) + return false; + + var wanted = (int)Math.Min(_buffer.Length, _limit - _position); + + while (_count < wanted) { - return Sequencer.None; + var read = _stream.Read(_buffer.AsSpan(_count, wanted - _count)); + if (read == 0) + break; + _count += read; } - return System.Buffers.Binary.BinaryPrimitives.ReadUInt64LittleEndian( - record.Payload.Slice(8, 8)); + return _count != 0; + } + + public void Dispose() + { + var buffer = _buffer; + _buffer = null; + if (buffer is not null) + ArrayPool.Shared.Return(buffer); + } + } + + private static JournalReadResult ReadSegmentHeader(string path, out SegmentDescriptor descriptor) + { + descriptor = default; + + if (!TryParseSegmentIndex(path, out var index)) + return JournalReadResult.SegmentOrder; + + try + { + using var stream = new FileStream(path, FileMode.Open, FileAccess.Read, + FileShare.ReadWrite | FileShare.Delete, bufferSize: 1, + FileOptions.SequentialScan); + var result = ReadRecord(stream, stream.Length, out var rented, out var size); + + if (result != JournalReadResult.Ok) + return result; + + try + { + JournalRecord.TryRead(rented.AsSpan(0, size), out var record); + + if (record.Type != JournalRecordType.SegmentHeader || record.Payload.Length != 16) + return JournalReadResult.SegmentOrder; + + var session = BinaryPrimitives.ReadUInt64LittleEndian(record.Payload); + if (session == 0) + return JournalReadResult.BadSession; + + descriptor = new SegmentDescriptor(path, index, session, + BinaryPrimitives.ReadUInt64LittleEndian(record.Payload.Slice(8)), size); + return JournalReadResult.Ok; + } + finally + { + ArrayPool.Shared.Return(rented); + } } catch (IOException) { - return Sequencer.None; + return JournalReadResult.Incomplete; } } - /// Handler invoked per record. Return false to stop the scan. + private static bool IsExpectedHeader(in JournalRecordView record, SegmentDescriptor segment) + => record.Type == JournalRecordType.SegmentHeader && record.TotalSize == segment.HeaderSize && + record.Payload.Length == 16 && + BinaryPrimitives.ReadUInt64LittleEndian(record.Payload) == segment.SessionId && + BinaryPrimitives.ReadUInt64LittleEndian(record.Payload.Slice(8)) == segment.FirstSequence; + + private static int SelectFirstSegment(List segments, ulong fromSequence) + { + var first = 0; + + if (fromSequence == Sequencer.None) + return first; + + for (var i = 0; i + 1 < segments.Count; i++) + { + if (segments[i + 1].FirstSequence <= fromSequence) + first = i + 1; + else + break; + } + + return first; + } + + private static bool TryParseSegmentIndex(string path, out int index) + { + const string prefix = "segment-"; + var name = Path.GetFileNameWithoutExtension(path); + index = -1; + + return name.StartsWith(prefix, StringComparison.Ordinal) && + int.TryParse(name.AsSpan(prefix.Length), out index) && index >= 0; + } + + private static RecoveryReport Clean(long records, ulong lastSequence, ulong lastCheckpoint, + long validBytes, ulong sessionId, ulong nextSequence, bool hasSequencedRecords) + => new(RecoveryOutcome.Clean, records, lastSequence, lastCheckpoint, validBytes, + null, JournalReadResult.Ok, sessionId, nextSequence, hasSequencedRecords, 0); + + private static RecoveryReport Corrupt(string path, JournalReadResult failure, ulong sessionId) + => new(RecoveryOutcome.Corrupt, 0, Sequencer.None, Sequencer.None, 0, path, failure, + sessionId, 0, false, 0); + + private readonly record struct SegmentDescriptor(string Path, int Index, ulong SessionId, + ulong FirstSequence, int HeaderSize); + public delegate bool RecordHandler(in JournalRecordView record); } - /// A recovered message, copied out of the log so it outlives the scan. - public sealed record SequencedPayload(ulong Sequence, long Timestamp, byte[] Payload); + public sealed record SequencedPayload(ulong Sequence, long Timestamp, byte[] Payload) + { + public ulong SessionId { get; init; } + public ushort MessageCount { get; init; } = 1; + } } diff --git a/Common/Durability/JournalRecord.cs b/Common/Durability/JournalRecord.cs index c90a58e..09a2238 100644 --- a/Common/Durability/JournalRecord.cs +++ b/Common/Durability/JournalRecord.cs @@ -4,56 +4,26 @@ namespace MarketData.Common.Durability { - /// What a journal record describes. public enum JournalRecordType : byte { Invalid = 0, - - /// A sequenced market data message. Message = 1, - - /// A checkpoint marker: state up to this sequence is captured elsewhere. Checkpoint = 2, - - /// Opens a segment; carries the session and the sequence the segment starts at. SegmentHeader = 3, - - /// An auditable non-market event (risk decision, kill switch, entitlement change). Audit = 4, + FeedPacket = 5, } - /// - /// Fixed-layout framing for one journal record. - /// - /// - /// - /// The layout exists to make a partial record detectable, which is the only thing that - /// matters after a crash. A process killed mid-append leaves a prefix of a record on disk, and - /// a reader that cannot tell that prefix from a complete record will either resurrect a message - /// that was never durable or silently truncate the log at the wrong place. - /// - /// - /// Three properties give that detection. The length precedes the payload, so a reader knows how - /// much to expect before it reads it. The CRC covers header and payload together, so a torn - /// write that happens to leave a plausible length still fails. And the trailing length repeats - /// the leading one, so the log can also be walked backwards from its tail - which is how - /// recovery finds the last complete record without scanning from the beginning. - /// - /// - /// CRC-32C rather than a hash: it is a corruption check, not a signature, and the hardware - /// instruction makes it cost roughly nothing next to the write itself. - /// - /// + /// CRC-32C-framed append record with a repeated trailing payload length. public static class JournalRecord { - /// Marks the start of a record. Also the resynchronisation point after damage. + /// Record magic. public const uint Magic = 0x4A524E31; // "JRN1" public const int HeaderSize = 32; public const int TrailerSize = 4; public const int OverheadSize = HeaderSize + TrailerSize; - /// Largest payload a single record may carry. public const int MaxPayloadSize = 1 << 20; // magic:4 type:1 reserved:3 length:4 sequence:8 timestamp:8 crc:4 = 32 @@ -63,14 +33,20 @@ public static class JournalRecord private const int TimestampOffset = 20; private const int CrcOffset = 28; - public static int SizeFor(int payloadLength) => OverheadSize + payloadLength; + public static int SizeFor(int payloadLength) + { + if ((uint)payloadLength > MaxPayloadSize) + throw new ArgumentOutOfRangeException(nameof(payloadLength)); + + return OverheadSize + payloadLength; + } /// Writes a complete record into . /// Bytes written. public static int Write(Span destination, JournalRecordType type, ulong sequence, long timestamp, ReadOnlySpan payload) { - if (type == JournalRecordType.Invalid) + if (type is <= JournalRecordType.Invalid or > JournalRecordType.FeedPacket) throw new ArgumentOutOfRangeException(nameof(type), type, "Records need a real type."); if (payload.Length > MaxPayloadSize) @@ -97,44 +73,25 @@ public static int Write(Span destination, JournalRecordType type, ulong se var crc = Crc32C.Compute(destination.Slice(0, CrcOffset), destination.Slice(HeaderSize, payload.Length)); BinaryPrimitives.WriteUInt32LittleEndian(destination.Slice(CrcOffset, 4), crc); - // Trailing length so the log can be walked backwards from the tail. BinaryPrimitives.WriteInt32LittleEndian(destination.Slice(HeaderSize + payload.Length, 4), payload.Length); return total; } - /// - /// Validates a record at the start of without copying it. - /// - /// - /// Returns rather than an error when the buffer - /// simply ends early. The distinction is the whole point: an incomplete tail is the normal - /// state of a log whose writer was killed, and is recoverable by truncation, whereas a - /// checksum failure in the middle of a log is corruption and is not. - /// + /// Validates one record without copying; a short buffer is Incomplete. public static JournalReadResult TryRead(ReadOnlySpan source, out JournalRecordView record) { record = default; - if (source.Length < HeaderSize) - return JournalReadResult.Incomplete; - - if (BinaryPrimitives.ReadUInt32LittleEndian(source) != Magic) - return JournalReadResult.BadMagic; - - var type = (JournalRecordType)source[TypeOffset]; - if (type == JournalRecordType.Invalid || type > JournalRecordType.Audit) - return JournalReadResult.BadType; - - var length = BinaryPrimitives.ReadInt32LittleEndian(source.Slice(LengthOffset, 4)); - if (length < 0 || length > MaxPayloadSize) - return JournalReadResult.BadLength; + var headerResult = TryGetSize(source, out var total); - var total = SizeFor(length); + if (headerResult != JournalReadResult.Ok) + return headerResult; if (source.Length < total) return JournalReadResult.Incomplete; + var length = total - OverheadSize; var stored = BinaryPrimitives.ReadUInt32LittleEndian(source.Slice(CrcOffset, 4)); var actual = Crc32C.Compute(source.Slice(0, CrcOffset), source.Slice(HeaderSize, length)); @@ -145,7 +102,7 @@ record = default; return JournalReadResult.BadTrailer; record = new JournalRecordView( - type, + (JournalRecordType)source[TypeOffset], BinaryPrimitives.ReadUInt64LittleEndian(source.Slice(SequenceOffset, 8)), BinaryPrimitives.ReadInt64LittleEndian(source.Slice(TimestampOffset, 8)), total, @@ -153,6 +110,31 @@ record = default; return JournalReadResult.Ok; } + + internal static JournalReadResult TryGetSize(ReadOnlySpan source, out int total) + { + total = 0; + + if (source.Length < HeaderSize) + return JournalReadResult.Incomplete; + + if (BinaryPrimitives.ReadUInt32LittleEndian(source) != Magic) + return JournalReadResult.BadMagic; + + var type = (JournalRecordType)source[TypeOffset]; + if (type is <= JournalRecordType.Invalid or > JournalRecordType.FeedPacket) + return JournalReadResult.BadType; + + if ((source[TypeOffset + 1] | source[TypeOffset + 2] | source[TypeOffset + 3]) != 0) + return JournalReadResult.BadFlags; + + var length = BinaryPrimitives.ReadInt32LittleEndian(source.Slice(LengthOffset, 4)); + if (length < 0 || length > MaxPayloadSize) + return JournalReadResult.BadLength; + + total = SizeFor(length); + return JournalReadResult.Ok; + } } public enum JournalReadResult : byte @@ -164,9 +146,13 @@ public enum JournalReadResult : byte BadMagic, BadType, + BadFlags, BadLength, BadChecksum, BadTrailer, + BadSession, + SequenceGap, + SegmentOrder, } /// A borrowed view over one record. Valid only while the source buffer is. @@ -186,7 +172,6 @@ public JournalRecordView(JournalRecordType type, ulong sequence, long timestamp, public ulong Sequence { get; } public long Timestamp { get; } - /// Bytes this record occupies, header and trailer included. public int TotalSize { get; } public ReadOnlySpan Payload { get; } diff --git a/Common/Durability/RetransmissionService.cs b/Common/Durability/RetransmissionService.cs index ead0b40..3c6f917 100644 --- a/Common/Durability/RetransmissionService.cs +++ b/Common/Durability/RetransmissionService.cs @@ -6,57 +6,77 @@ using System.Net.Sockets; using System.Threading; using System.Threading.Tasks; +using MarketData.Common.Feed; namespace MarketData.Common.Durability { - /// - /// Serves gap-fill requests out of the journal, over TCP. - /// - /// - /// - /// Deliberately a separate channel from the live feed, and deliberately reliable where the - /// feed is not. This mirrors how real venues do it, and the reason is not tradition: a - /// subscriber asking for a retransmission has already lost packets, so answering on the same - /// lossy multicast group it just lost them on is a poor bet. TCP also gives per-subscriber - /// flow control, which matters because a recovering subscriber wants a burst of history while - /// everyone else wants the live feed uninterrupted. - /// - /// - /// The service is intentionally cheap to refuse. Retransmission is where a struggling - /// subscriber can turn a local problem into a publisher-wide one, so a request for more than - /// messages is rejected outright rather than served slowly. A - /// subscriber that far behind should recover from a snapshot instead, which is O(book) rather - /// than O(history). - /// - /// + public enum RetransmissionStatus : ushort + { + Success = 0, + SnapshotRequired = 1, + WrongSession = 2, + InvalidRequest = 3, + CorruptJournal = 4, + } + + /// Bounded TCP gap-fill service over an exact journal prefix. public sealed class RetransmissionService : IDisposable { - /// Largest gap that will be filled from history rather than by snapshot. + public const uint Magic = 0x32585452; // RTX2 + public const ushort Version = 2; public const int MaxRangeLength = 10_000; - - public const int RequestSize = 16; + public const int RequestSize = 36; + public const int ResponseSize = 24; + public const int FrameHeaderSize = 20; + public static readonly TimeSpan RequestTimeout = TimeSpan.FromSeconds(5); private readonly TcpListener _listener; - private readonly string _journalDirectory; + private readonly ulong _sessionId; + private readonly JournalRangeReader _rangeReader; private readonly CancellationTokenSource _shutdown = new(); + private readonly SemaphoreSlim _slots; + private readonly object _clientGate = new(); + private readonly HashSet _clients = new(); + private Task _accepting; + private long _requestsServed; + private long _requestsRefused; + private long _messagesSent; + private int _started; private int _disposed; - public int Port => ((IPEndPoint)_listener.LocalEndpoint).Port; - - public long RequestsServed { get; private set; } - public long RequestsRefused { get; private set; } - public long MessagesSent { get; private set; } - - public RetransmissionService(string journalDirectory, int port = 0, IPAddress address = null) + public RetransmissionService(string journalDirectory, int port = 0, + IPAddress address = null, int maxConcurrentRequests = 8) { - _journalDirectory = journalDirectory; + ArgumentException.ThrowIfNullOrWhiteSpace(journalDirectory); + if ((uint)port > ushort.MaxValue) + throw new ArgumentOutOfRangeException(nameof(port)); + if (maxConcurrentRequests is < 1 or > 1024) + throw new ArgumentOutOfRangeException(nameof(maxConcurrentRequests)); + + var report = JournalReader.Recover(journalDirectory); + if (report.Outcome == RecoveryOutcome.Corrupt || report.SessionId == 0) + throw new InvalidDataException("Retransmission requires a valid journal session."); + + _sessionId = report.SessionId; + _rangeReader = new JournalRangeReader(journalDirectory); + _slots = new SemaphoreSlim(maxConcurrentRequests, maxConcurrentRequests); _listener = new TcpListener(address ?? IPAddress.Loopback, port); } + public int Port => ((IPEndPoint)_listener.LocalEndpoint).Port; + public ulong SessionId => _sessionId; + public long RequestsServed => Interlocked.Read(ref _requestsServed); + public long RequestsRefused => Interlocked.Read(ref _requestsRefused); + public long MessagesSent => Interlocked.Read(ref _messagesSent); + public void Start() { - _listener.Start(); + ObjectDisposedException.ThrowIf(Volatile.Read(ref _disposed) != 0, this); + if (Interlocked.Exchange(ref _started, 1) != 0) + throw new InvalidOperationException("Retransmission service is already running."); + + _listener.Start(backlog: 128); _accepting = AcceptLoopAsync(_shutdown.Token); } @@ -64,103 +84,187 @@ private async Task AcceptLoopAsync(CancellationToken token) { while (!token.IsCancellationRequested) { - TcpClient client; - try { - client = await _listener.AcceptTcpClientAsync(token).ConfigureAwait(false); + await _slots.WaitAsync(token).ConfigureAwait(false); + TcpClient client; + + try + { + client = await _listener.AcceptTcpClientAsync(token).ConfigureAwait(false); + } + catch + { + _slots.Release(); + throw; + } + + var task = ServeAndReleaseAsync(client, token); + lock (_clientGate) + _clients.Add(task); + _ = RemoveWhenCompleteAsync(task); } catch (OperationCanceledException) { return; } - catch (SocketException) + catch (SocketException) when (token.IsCancellationRequested) { return; } + } + } - // Each request is served on its own task so one slow recovering subscriber cannot - // block the queue behind it. - _ = ServeAsync(client, token); + private async Task RemoveWhenCompleteAsync(Task task) + { + try { await task.ConfigureAwait(false); } + finally + { + lock (_clientGate) + _clients.Remove(task); } } - private async Task ServeAsync(TcpClient client, CancellationToken token) + private async Task ServeAndReleaseAsync(TcpClient client, CancellationToken serviceToken) { - using (client) + try { - try + using (client) + using (var timeout = CancellationTokenSource.CreateLinkedTokenSource(serviceToken)) { - client.NoDelay = true; - var stream = client.GetStream(); + timeout.CancelAfter(RequestTimeout); + await ServeAsync(client, timeout.Token).ConfigureAwait(false); + } + } + catch (Exception error) when (error is IOException or SocketException or + OperationCanceledException) + { + // A failed recovery connection does not affect the live feed. + } + finally + { + _slots.Release(); + } + } - var request = new byte[RequestSize]; - await ReadExactlyAsync(stream, request, token).ConfigureAwait(false); + private async Task ServeAsync(TcpClient client, CancellationToken token) + { + client.NoDelay = true; + var stream = client.GetStream(); + var request = new byte[RequestSize]; + await ReadExactlyAsync(stream, request, token).ConfigureAwait(false); + + if (BinaryPrimitives.ReadUInt32LittleEndian(request) != Magic || + BinaryPrimitives.ReadUInt16LittleEndian(request.AsSpan(4)) != Version || + BinaryPrimitives.ReadUInt16LittleEndian(request.AsSpan(6)) != 0 || + BinaryPrimitives.ReadUInt32LittleEndian(request.AsSpan(32)) != + Crc32C.Compute(request.AsSpan(0, 32))) + { + await RefuseAsync(stream, RetransmissionStatus.InvalidRequest, _sessionId, token) + .ConfigureAwait(false); + return; + } - var from = BinaryPrimitives.ReadUInt64LittleEndian(request); - var to = BinaryPrimitives.ReadUInt64LittleEndian(request.AsSpan(8)); + var requestedSession = BinaryPrimitives.ReadUInt64LittleEndian(request.AsSpan(8)); + var from = BinaryPrimitives.ReadUInt64LittleEndian(request.AsSpan(16)); + var to = BinaryPrimitives.ReadUInt64LittleEndian(request.AsSpan(24)); - if (to < from || to - from + 1 > MaxRangeLength) - { - RequestsRefused++; - await WriteRefusalAsync(stream, token).ConfigureAwait(false); - return; - } + if (requestedSession != 0 && requestedSession != _sessionId) + { + await RefuseAsync(stream, RetransmissionStatus.WrongSession, _sessionId, token) + .ConfigureAwait(false); + return; + } - var messages = JournalReader.ReadRange(_journalDirectory, from, to); - RequestsServed++; + if (to < from || to - from >= MaxRangeLength) + { + await RefuseAsync(stream, RetransmissionStatus.InvalidRequest, _sessionId, token) + .ConfigureAwait(false); + return; + } - var count = new byte[4]; - BinaryPrimitives.WriteInt32LittleEndian(count, messages.Count); - await stream.WriteAsync(count, token).ConfigureAwait(false); + JournalRangeResult result; + List messages; - foreach (var message in messages) - { - var frame = new byte[12]; - BinaryPrimitives.WriteUInt64LittleEndian(frame, message.Sequence); - BinaryPrimitives.WriteInt32LittleEndian(frame.AsSpan(8), message.Payload.Length); + try + { + result = _rangeReader.TryRead(_sessionId, from, to, out messages); + } + catch (Exception error) when (error is InvalidDataException or IOException) + { + await RefuseAsync(stream, RetransmissionStatus.CorruptJournal, _sessionId, token) + .ConfigureAwait(false); + return; + } - await stream.WriteAsync(frame, token).ConfigureAwait(false); - await stream.WriteAsync(message.Payload, token).ConfigureAwait(false); - MessagesSent++; - } + var status = result switch + { + JournalRangeResult.Success => RetransmissionStatus.Success, + JournalRangeResult.WrongSession => RetransmissionStatus.WrongSession, + JournalRangeResult.Corrupt => RetransmissionStatus.CorruptJournal, + _ => RetransmissionStatus.SnapshotRequired, + }; - await stream.FlushAsync(token).ConfigureAwait(false); - } - catch (Exception) when (token.IsCancellationRequested) - { - // Shutdown, not a failure. - } - catch (IOException) - { - // The recovering subscriber gave up. Its problem, not ours. - } - catch (OperationCanceledException) - { - } + if (status != RetransmissionStatus.Success) + { + await RefuseAsync(stream, status, _sessionId, token).ConfigureAwait(false); + return; } + + Interlocked.Increment(ref _requestsServed); + await WriteResponseHeaderAsync(stream, status, _sessionId, messages.Count, token) + .ConfigureAwait(false); + + foreach (var message in messages) + { + var frame = new byte[FrameHeaderSize]; + BinaryPrimitives.WriteUInt64LittleEndian(frame, message.Sequence); + BinaryPrimitives.WriteUInt16LittleEndian(frame.AsSpan(8), message.MessageCount); + BinaryPrimitives.WriteUInt16LittleEndian(frame.AsSpan(10), 0); + BinaryPrimitives.WriteInt32LittleEndian(frame.AsSpan(12), message.Payload.Length); + BinaryPrimitives.WriteUInt32LittleEndian(frame.AsSpan(16), + Crc32C.Compute(frame.AsSpan(0, 16), message.Payload)); + + await stream.WriteAsync(frame, token).ConfigureAwait(false); + await stream.WriteAsync(message.Payload, token).ConfigureAwait(false); + Interlocked.Increment(ref _messagesSent); + } + + await stream.FlushAsync(token).ConfigureAwait(false); } - /// A refusal is an empty range: "recover from a snapshot instead". - private static async Task WriteRefusalAsync(NetworkStream stream, CancellationToken token) + private async Task RefuseAsync(NetworkStream stream, RetransmissionStatus status, + ulong sessionId, CancellationToken token) { - var count = new byte[4]; - BinaryPrimitives.WriteInt32LittleEndian(count, -1); - await stream.WriteAsync(count, token).ConfigureAwait(false); + Interlocked.Increment(ref _requestsRefused); + await WriteResponseHeaderAsync(stream, status, sessionId, 0, token).ConfigureAwait(false); await stream.FlushAsync(token).ConfigureAwait(false); } - internal static async Task ReadExactlyAsync(Stream stream, Memory buffer, CancellationToken token) + private static async Task WriteResponseHeaderAsync(NetworkStream stream, + RetransmissionStatus status, ulong sessionId, int count, CancellationToken token) + { + var response = new byte[ResponseSize]; + BinaryPrimitives.WriteUInt32LittleEndian(response, Magic); + BinaryPrimitives.WriteUInt16LittleEndian(response.AsSpan(4), Version); + BinaryPrimitives.WriteUInt16LittleEndian(response.AsSpan(6), (ushort)status); + BinaryPrimitives.WriteUInt64LittleEndian(response.AsSpan(8), sessionId); + BinaryPrimitives.WriteInt32LittleEndian(response.AsSpan(16), count); + BinaryPrimitives.WriteUInt32LittleEndian(response.AsSpan(20), + Crc32C.Compute(response.AsSpan(0, 20))); + await stream.WriteAsync(response, token).ConfigureAwait(false); + } + + internal static async Task ReadExactlyAsync(Stream stream, Memory buffer, + CancellationToken token) { var read = 0; while (read < buffer.Length) { var got = await stream.ReadAsync(buffer.Slice(read), token).ConfigureAwait(false); - if (got == 0) - throw new IOException("Peer closed before the message was complete."); - + throw new IOException("Peer closed a partial frame."); read += got; } } @@ -171,28 +275,26 @@ public void Dispose() return; _shutdown.Cancel(); + _listener.Stop(); - try - { - _listener.Stop(); - } - catch (SocketException) - { - } + try { _accepting?.Wait(TimeSpan.FromSeconds(5)); } + catch (AggregateException) { } - try - { - _accepting?.Wait(TimeSpan.FromSeconds(5)); - } - catch (AggregateException) - { - } + Task[] clients; + lock (_clientGate) + clients = new List(_clients).ToArray(); + try { Task.WaitAll(clients, TimeSpan.FromSeconds(5)); } + catch (AggregateException) { } + + _slots.Dispose(); _shutdown.Dispose(); } } - /// Client side of gap fill. + public sealed record RetransmissionResponse(RetransmissionStatus Status, ulong SessionId, + List Messages); + public sealed class RetransmissionClient { private readonly IPEndPoint _endpoint; @@ -200,55 +302,106 @@ public sealed class RetransmissionClient public RetransmissionClient(int port, IPAddress address = null) => _endpoint = new IPEndPoint(address ?? IPAddress.Loopback, port); - /// - /// Requests [from, to]. - /// - /// - /// The recovered messages, or null when the publisher refused - which means the gap is too - /// large to fill from history and the subscriber should wait for the next snapshot. - /// public async Task> RequestAsync(ulong from, ulong to, CancellationToken token = default) { + var response = await RequestDetailedAsync(0, from, to, token).ConfigureAwait(false); + return response.Status == RetransmissionStatus.Success ? response.Messages : null; + } + + public async Task> RequestAsync(ulong sessionId, ulong from, ulong to, + CancellationToken token = default) + { + var response = await RequestDetailedAsync(sessionId, from, to, token).ConfigureAwait(false); + return response.Status == RetransmissionStatus.Success ? response.Messages : null; + } + + public async Task RequestDetailedAsync(ulong sessionId, ulong from, + ulong to, CancellationToken token = default) + { + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(token); + timeout.CancelAfter(RetransmissionService.RequestTimeout); + token = timeout.Token; + using var client = new TcpClient(); await client.ConnectAsync(_endpoint, token).ConfigureAwait(false); client.NoDelay = true; - var stream = client.GetStream(); var request = new byte[RetransmissionService.RequestSize]; - BinaryPrimitives.WriteUInt64LittleEndian(request, from); - BinaryPrimitives.WriteUInt64LittleEndian(request.AsSpan(8), to); + BinaryPrimitives.WriteUInt32LittleEndian(request, RetransmissionService.Magic); + BinaryPrimitives.WriteUInt16LittleEndian(request.AsSpan(4), RetransmissionService.Version); + BinaryPrimitives.WriteUInt16LittleEndian(request.AsSpan(6), 0); + BinaryPrimitives.WriteUInt64LittleEndian(request.AsSpan(8), sessionId); + BinaryPrimitives.WriteUInt64LittleEndian(request.AsSpan(16), from); + BinaryPrimitives.WriteUInt64LittleEndian(request.AsSpan(24), to); + BinaryPrimitives.WriteUInt32LittleEndian(request.AsSpan(32), + Crc32C.Compute(request.AsSpan(0, 32))); await stream.WriteAsync(request, token).ConfigureAwait(false); - await stream.FlushAsync(token).ConfigureAwait(false); - var countBuffer = new byte[4]; - await RetransmissionService.ReadExactlyAsync(stream, countBuffer, token).ConfigureAwait(false); - var count = BinaryPrimitives.ReadInt32LittleEndian(countBuffer); + var header = new byte[RetransmissionService.ResponseSize]; + await RetransmissionService.ReadExactlyAsync(stream, header, token).ConfigureAwait(false); + + if (BinaryPrimitives.ReadUInt32LittleEndian(header) != RetransmissionService.Magic || + BinaryPrimitives.ReadUInt16LittleEndian(header.AsSpan(4)) != + RetransmissionService.Version || + BinaryPrimitives.ReadUInt32LittleEndian(header.AsSpan(20)) != + Crc32C.Compute(header.AsSpan(0, 20))) + throw new IOException("Retransmission response header failed validation."); - if (count < 0) - return null; + var status = (RetransmissionStatus)BinaryPrimitives.ReadUInt16LittleEndian(header.AsSpan(6)); + var responseSession = BinaryPrimitives.ReadUInt64LittleEndian(header.AsSpan(8)); + var count = BinaryPrimitives.ReadInt32LittleEndian(header.AsSpan(16)); + + if (!Enum.IsDefined(status) || count < 0 || count > RetransmissionService.MaxRangeLength || + responseSession == 0 || + (status != RetransmissionStatus.Success && count != 0) || + (status == RetransmissionStatus.Success && count == 0) || + (sessionId != 0 && responseSession != sessionId && + status != RetransmissionStatus.WrongSession)) + throw new IOException("Retransmission response is inconsistent."); var messages = new List(count); + var cursor = from; + var coveredThrough = from; for (var i = 0; i < count; i++) { - var frame = new byte[12]; + var frame = new byte[RetransmissionService.FrameHeaderSize]; await RetransmissionService.ReadExactlyAsync(stream, frame, token).ConfigureAwait(false); - var sequence = BinaryPrimitives.ReadUInt64LittleEndian(frame); - var length = BinaryPrimitives.ReadInt32LittleEndian(frame.AsSpan(8)); + var messageCount = BinaryPrimitives.ReadUInt16LittleEndian(frame.AsSpan(8)); + var flags = BinaryPrimitives.ReadUInt16LittleEndian(frame.AsSpan(10)); + var length = BinaryPrimitives.ReadInt32LittleEndian(frame.AsSpan(12)); + var storedCrc = BinaryPrimitives.ReadUInt32LittleEndian(frame.AsSpan(16)); + + if (messageCount == 0 || flags != 0 || length < 0 || + length > JournalRecord.MaxPayloadSize || sequence != cursor || + sequence > ulong.MaxValue - messageCount) + throw new IOException("Retransmission frame is invalid."); - if (length < 0 || length > JournalRecord.MaxPayloadSize) - throw new IOException($"Retransmission declared an implausible length of {length}."); + coveredThrough = sequence + messageCount - 1; + if (coveredThrough > to) + throw new IOException("Retransmission response exceeds the requested range."); var payload = new byte[length]; await RetransmissionService.ReadExactlyAsync(stream, payload, token).ConfigureAwait(false); + if (storedCrc != Crc32C.Compute(frame.AsSpan(0, 16), payload)) + throw new IOException("Retransmission frame checksum failed."); - messages.Add(new SequencedPayload(sequence, 0, payload)); + messages.Add(new SequencedPayload(sequence, 0, payload) + { + SessionId = responseSession, + MessageCount = messageCount, + }); + + cursor = coveredThrough == ulong.MaxValue ? ulong.MaxValue : coveredThrough + 1; } - return messages; + if (status == RetransmissionStatus.Success && coveredThrough != to) + throw new IOException("Retransmission response does not cover the requested range."); + + return new RetransmissionResponse(status, responseSession, messages); } } } diff --git a/Common/Durability/Sequencer.cs b/Common/Durability/Sequencer.cs index c229718..4ef0cae 100644 --- a/Common/Durability/Sequencer.cs +++ b/Common/Durability/Sequencer.cs @@ -3,74 +3,60 @@ namespace MarketData.Common.Durability { - /// - /// Assigns the single global order that every downstream consumer agrees on. - /// - /// - /// - /// The sequence number is the contract. A subscriber detects loss because numbers skip, a - /// backup takes over at a known point because the number is durable, and a retransmission - /// request is expressible at all because ranges are named by it. Everything else in this - /// namespace exists to keep that number meaningful. - /// - /// - /// Sequences start at 1, so 0 is available as "nothing yet" without a nullable and without a - /// sentinel that could collide with a real value. - /// - /// + /// Lock-free monotonic sequence allocator. Sequence zero is the empty watermark. public sealed class Sequencer { - /// The value meaning "no sequence has been assigned". public const ulong None = 0; - private long _last; + // Interlocked has signed overloads; the bits remain an unsigned counter. + private long _lastBits; - public Sequencer(ulong resumeFrom = None) => _last = (long)resumeFrom; + public Sequencer(ulong resumeFrom = None) => _lastBits = unchecked((long)resumeFrom); - /// The most recently assigned sequence, or . - public ulong Last => (ulong)Interlocked.Read(ref _last); + public ulong Last => unchecked((ulong)Interlocked.Read(ref _lastBits)); - /// Assigns the next sequence. - /// - /// Interlocked rather than a plain increment because the sequencer is the one component - /// several producers legitimately share - unlike the ring buffers downstream of it, which - /// are single-producer by construction. - /// - public ulong Next() => (ulong)Interlocked.Increment(ref _last); + public ulong Next() => Reserve(1); - /// - /// Reserves consecutive sequences and returns the first. - /// - /// - /// A batch published in one packet must occupy a contiguous range, or a subscriber that - /// receives the packet would see a gap that never existed. - /// + /// Reserves a contiguous range and returns its first sequence. public ulong Reserve(int count) { if (count <= 0) throw new ArgumentOutOfRangeException(nameof(count), count, "Must reserve at least one."); - var last = Interlocked.Add(ref _last, count); - return (ulong)last - (ulong)count + 1; + while (true) + { + var observedBits = Interlocked.Read(ref _lastBits); + var observed = unchecked((ulong)observedBits); + + if (observed > ulong.MaxValue - (uint)count) + throw new OverflowException("The sequence space is exhausted."); + + var next = observed + (uint)count; + var nextBits = unchecked((long)next); + + if (Interlocked.CompareExchange(ref _lastBits, nextBits, observedBits) == observedBits) + return observed + 1; + } } - /// - /// Resumes from a recovered watermark, refusing to move backwards. - /// - /// - /// Rewinding a sequencer re-issues numbers that subscribers have already seen and applied, - /// which is indistinguishable to them from a stuck feed and corrupts every book downstream. - /// It is rejected rather than clamped so the caller finds out. - /// + /// Advances to a recovered watermark; rewinds fail. public void ResumeFrom(ulong sequence) { - var current = Last; - - if (sequence < current) - throw new ArgumentOutOfRangeException(nameof(sequence), sequence, - $"Refusing to rewind the sequencer from {current}; numbers would be reissued."); - - Interlocked.Exchange(ref _last, (long)sequence); + while (true) + { + var observedBits = Interlocked.Read(ref _lastBits); + var observed = unchecked((ulong)observedBits); + + if (sequence < observed) + throw new ArgumentOutOfRangeException(nameof(sequence), sequence, + $"Cannot rewind from {observed}."); + if (sequence == observed) + return; + + if (Interlocked.CompareExchange(ref _lastBits, unchecked((long)sequence), observedBits) == + observedBits) + return; + } } } } diff --git a/Common/Durability/WriteAheadJournal.cs b/Common/Durability/WriteAheadJournal.cs index 5362e01..f696f6a 100644 --- a/Common/Durability/WriteAheadJournal.cs +++ b/Common/Durability/WriteAheadJournal.cs @@ -1,254 +1,454 @@ using System; using System.Buffers; +using System.Buffers.Binary; using System.Collections.Generic; +using System.Collections.Concurrent; using System.IO; using System.Linq; using System.Threading; +using MarketData.Common.Feed; namespace MarketData.Common.Durability { - /// When an append is considered durable. - /// - /// This is the honest name for a trade-off that is usually hidden. Every option below loses - /// data under some failure; they differ in which failure. - /// public enum DurabilityPolicy { - /// - /// Hand the bytes to the OS and return. Survives process death, loses whatever the page - /// cache held if the machine dies. - /// OsBuffered, - - /// - /// fsync every append. Survives machine death up to the last returned append, and costs a - /// device round trip per record. - /// SyncEachRecord, - - /// - /// fsync at most every . Bounds the loss - /// window by time rather than eliminating it - the usual choice, and the one that must - /// state its window out loud. - /// SyncPeriodic, } - /// - /// An append-only, CRC-checked, segmented log of sequenced records. - /// - /// - /// - /// Write-ahead means what it says: a message is journalled before it is published. A subscriber - /// can then never hold a message the publisher cannot reproduce, which is what makes - /// retransmission and backup takeover possible at all. Publish-then-journal would invert that - /// and create messages that exist only in flight. - /// - /// - /// Segments rather than one file, because retention and recovery both work in whole files: - /// old segments are deleted without rewriting anything, and a damaged segment bounds the - /// damage. Each segment opens with a header record naming the session and its first sequence, - /// so a segment is self-describing and recovery does not depend on file names. - /// - /// + /// Single-writer, segmented WAL with strict sequence and crash-tail recovery. public sealed class WriteAheadJournal : IDisposable { - /// How often reaches the device. public static readonly TimeSpan FlushInterval = TimeSpan.FromMilliseconds(200); private const string SegmentPrefix = "segment-"; private const string SegmentSuffix = ".jrn"; + private const string WriterLeaseName = "writer.lock"; + private const int StackRecordLimit = 2048; + private static readonly int SegmentHeaderBytes = JournalRecord.SizeFor(16); + private static readonly ConcurrentDictionary ActiveWriters = + new(StringComparer.Ordinal); private readonly object _gate = new(); private readonly string _directory; private readonly long _segmentBytes; - private readonly ulong _sessionId; + private readonly ulong _initialSequence; + private readonly string _leaseKey; + private readonly FileStream _writerLease; + private readonly ManualResetEventSlim _syncShutdown; + private readonly Thread _syncThread; private FileStream _segment; private long _segmentLength; private int _segmentIndex; - private long _lastFlushTicks; + private ulong _nextSequence; + private long _recordsAppended; + private long _syncs; + private Exception _failure; + private bool _hasSequencedRecords; + private bool _dirty; private bool _disposed; + public WriteAheadJournal(string directory, ulong sessionId, + DurabilityPolicy policy = DurabilityPolicy.SyncPeriodic, + long segmentBytes = 64L * 1024 * 1024, + TimeSpan? syncInterval = null, + ulong initialSequence = 1) + { + ArgumentException.ThrowIfNullOrWhiteSpace(directory); + if (sessionId == 0) + throw new ArgumentOutOfRangeException(nameof(sessionId)); + if (!Enum.IsDefined(policy)) + throw new ArgumentOutOfRangeException(nameof(policy)); + if (segmentBytes < SegmentHeaderBytes + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize)) + throw new ArgumentOutOfRangeException(nameof(segmentBytes)); + + var interval = syncInterval ?? FlushInterval; + if (interval <= TimeSpan.Zero || interval.TotalMilliseconds > uint.MaxValue - 1) + throw new ArgumentOutOfRangeException(nameof(syncInterval)); + + _directory = Path.GetFullPath(directory); + _leaseKey = OperatingSystem.IsWindows() ? _directory.ToUpperInvariant() : _directory; + _segmentBytes = segmentBytes; + _initialSequence = initialSequence; + _nextSequence = initialSequence; + SessionId = sessionId; + Policy = policy; + SyncInterval = interval; + + Directory.CreateDirectory(_directory); + + if (!ActiveWriters.TryAdd(_leaseKey, 0)) + throw new IOException("The journal already has a writer in this process."); + + try + { + _writerLease = new FileStream(Path.Combine(_directory, WriterLeaseName), + FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None, bufferSize: 1); + OpenOrRecover(); + + if (policy == DurabilityPolicy.SyncPeriodic) + { + _syncShutdown = new ManualResetEventSlim(false); + _syncThread = new Thread(SyncLoop) + { + IsBackground = true, + Name = $"journal-sync-{sessionId:X16}", + }; + _syncThread.Start(); + } + } + catch + { + _segment?.Dispose(); + _writerLease?.Dispose(); + _syncShutdown?.Dispose(); + ActiveWriters.TryRemove(_leaseKey, out _); + throw; + } + } + public DurabilityPolicy Policy { get; } + public TimeSpan SyncInterval { get; } + public ulong SessionId { get; } + public string DirectoryPath => _directory; + + public ulong LastSequence + { + get { lock (_gate) return _hasSequencedRecords ? _nextSequence - 1 : Sequencer.None; } + } - /// Highest sequence written, or . - public ulong LastSequence { get; private set; } + public ulong NextSequence + { + get { lock (_gate) return _nextSequence; } + } - /// Records appended since this journal was opened. - public long RecordsAppended { get; private set; } + public bool HasSequencedRecords + { + get { lock (_gate) return _hasSequencedRecords; } + } - /// Times the log has actually been forced to the device. - public long Syncs { get; private set; } + public long RecordsAppended => Interlocked.Read(ref _recordsAppended); + public long Syncs => Interlocked.Read(ref _syncs); - public WriteAheadJournal(string directory, ulong sessionId, - DurabilityPolicy policy = DurabilityPolicy.SyncPeriodic, long segmentBytes = 64L * 1024 * 1024) + public ulong Append(JournalRecordType type, ulong sequence, long timestamp, + ReadOnlySpan payload) { - if (sessionId == 0) - throw new ArgumentOutOfRangeException(nameof(sessionId), "Session id 0 is reserved."); - if (segmentBytes < JournalRecord.OverheadSize + JournalRecord.MaxPayloadSize) - throw new ArgumentOutOfRangeException(nameof(segmentBytes), - "A segment must be able to hold at least one maximum-sized record."); + if (type is JournalRecordType.Invalid or JournalRecordType.SegmentHeader or + JournalRecordType.FeedPacket || type > JournalRecordType.FeedPacket) + throw new ArgumentOutOfRangeException(nameof(type)); - _directory = directory; - _sessionId = sessionId; - _segmentBytes = segmentBytes; - Policy = policy; + return AppendCore(type, sequence, timestamp, payload, 1); + } - Directory.CreateDirectory(directory); + public ulong AppendNext(long timestamp, ReadOnlySpan payload) + { + lock (_gate) + { + EnsureWritable(); + return AppendLocked(JournalRecordType.Message, _nextSequence, timestamp, payload, 1); + } + } + + /// Persists one sealed feed packet before multicast publication. + public ulong AppendPacket(ReadOnlySpan packet) + { + if (!FeedProtocol.TryReadHeader(packet, out var header, out var error)) + throw new InvalidDataException($"Invalid feed packet: {error}."); + if (header.SessionId != SessionId) + throw new InvalidDataException("Feed and journal sessions differ."); + + return AppendCore(JournalRecordType.FeedPacket, header.FirstSequence, + header.SourceTimestamp, packet, header.MessageCount); + } - var existing = SegmentFiles(directory); - _segmentIndex = existing.Count == 0 ? 0 : IndexOf(existing[^1]) + 1; + public void Sync() + { + lock (_gate) + { + EnsureWritable(); + ForceToDevice(); + } + } - OpenSegment(); + private ulong AppendCore(JournalRecordType type, ulong sequence, long timestamp, + ReadOnlySpan payload, ushort sequenceCount) + { + lock (_gate) + { + EnsureWritable(); + return AppendLocked(type, sequence, timestamp, payload, sequenceCount); + } } - /// Appends one record and returns the sequence it was written under. - public ulong Append(JournalRecordType type, ulong sequence, long timestamp, ReadOnlySpan payload) + private ulong AppendLocked(JournalRecordType type, ulong sequence, long timestamp, + ReadOnlySpan payload, ushort sequenceCount) { + ValidateSequence(type, sequence, sequenceCount, payload); + var size = JournalRecord.SizeFor(payload.Length); - var buffer = ArrayPool.Shared.Rent(size); + byte[] rented = null; + Span buffer = size <= StackRecordLimit + ? stackalloc byte[size] + : (rented = ArrayPool.Shared.Rent(size)); try { JournalRecord.Write(buffer, type, sequence, timestamp, payload); - lock (_gate) - { - ObjectDisposedException.ThrowIf(_disposed, this); - - // Rotate before writing, never mid-record: a record split across two segments - // could not be validated by either. - if (_segmentLength + size > _segmentBytes) - RollSegment(); - - _segment.Write(buffer, 0, size); - _segmentLength += size; - RecordsAppended++; + if (_segmentLength + size > _segmentBytes) + RollSegment(); - if (sequence > LastSequence) - LastSequence = sequence; + _segment.Write(buffer.Slice(0, size)); + _segmentLength += size; + _dirty = true; + Interlocked.Increment(ref _recordsAppended); - ApplyDurabilityPolicy(); + if (type is JournalRecordType.Message or JournalRecordType.FeedPacket) + { + _nextSequence = checked(sequence + sequenceCount); + _hasSequencedRecords = true; } + ApplyDurabilityPolicy(); return sequence; } + catch (Exception error) when (error is IOException or UnauthorizedAccessException) + { + _failure ??= error; + throw; + } finally { - ArrayPool.Shared.Return(buffer); + if (rented is not null) + ArrayPool.Shared.Return(rented); } } - /// Forces everything appended so far to the device. - public void Sync() + private void ValidateSequence(JournalRecordType type, ulong sequence, ushort sequenceCount, + ReadOnlySpan payload) { - lock (_gate) + switch (type) { - if (_disposed) - return; + case JournalRecordType.Message: + case JournalRecordType.FeedPacket: + if (sequence != _nextSequence) + throw new InvalidOperationException( + $"Expected sequence {_nextSequence}, received {sequence}."); + if (sequenceCount == 0 || sequence > ulong.MaxValue - sequenceCount) + throw new OverflowException("The journal sequence space is exhausted."); + break; - ForceToDevice(); + case JournalRecordType.Checkpoint: + if (payload.Length != sizeof(ulong) || + BinaryPrimitives.ReadUInt64LittleEndian(payload) != sequence || + (_hasSequencedRecords ? sequence > _nextSequence - 1 : sequence != Sequencer.None)) + throw new InvalidDataException("Checkpoint marker is outside the durable prefix."); + break; + + case JournalRecordType.Audit: + if (_hasSequencedRecords ? sequence > _nextSequence - 1 : sequence != Sequencer.None) + throw new InvalidDataException("Audit record is outside the durable prefix."); + break; } } private void ApplyDurabilityPolicy() { - switch (Policy) - { - case DurabilityPolicy.SyncEachRecord: - ForceToDevice(); - break; + if (Policy == DurabilityPolicy.SyncEachRecord) + ForceToDevice(); + } - case DurabilityPolicy.SyncPeriodic: - var now = Environment.TickCount64; - if (now - _lastFlushTicks >= FlushInterval.TotalMilliseconds) - ForceToDevice(); - break; + private void SyncLoop() + { + while (!_syncShutdown.Wait(SyncInterval)) + { + try + { + lock (_gate) + { + if (_disposed || _failure is not null) + return; + if (_dirty) + ForceToDevice(); + } + } + catch (Exception error) when (error is IOException or UnauthorizedAccessException) + { + lock (_gate) + _failure ??= error; - case DurabilityPolicy.OsBuffered: - // Deliberately nothing. The bytes are in the page cache and survive this - // process dying, which is the guarantee this policy offers and the only one. - break; + return; + } } } private void ForceToDevice() { - // flushToDisk: true is the part that matters. Stream.Flush() alone only moves bytes - // from the managed buffer into the OS, which is not durability, and is the usual way - // a journal turns out not to be one. + if (!_dirty) + return; + _segment.Flush(flushToDisk: true); - _lastFlushTicks = Environment.TickCount64; - Syncs++; + _dirty = false; + Interlocked.Increment(ref _syncs); } private void RollSegment() { ForceToDevice(); _segment.Dispose(); + + if (_segmentIndex == int.MaxValue) + throw new OverflowException("The segment index is exhausted."); + _segmentIndex++; - OpenSegment(); + OpenNewSegment(); } - private void OpenSegment() + private void OpenOrRecover() { - var path = Path.Combine(_directory, $"{SegmentPrefix}{_segmentIndex:D9}{SegmentSuffix}"); + var segments = SegmentFiles(_directory); - _segment = new FileStream(path, FileMode.Create, FileAccess.Write, FileShare.Read, - bufferSize: 64 * 1024, FileOptions.SequentialScan); - _segmentLength = 0; - _lastFlushTicks = Environment.TickCount64; + if (segments.Count == 0) + { + _segmentIndex = 0; + OpenNewSegment(); + return; + } - // A segment names itself, so recovery never has to trust a file name. - Span header = stackalloc byte[16]; - System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(header, _sessionId); - System.Buffers.Binary.BinaryPrimitives.WriteUInt64LittleEndian(header.Slice(8), - LastSequence + 1); + var report = JournalReader.Recover(_directory, expectedSessionId: SessionId, + expectedInitialSequence: _initialSequence); - var size = JournalRecord.SizeFor(header.Length); - var buffer = ArrayPool.Shared.Rent(size); + if (report.Outcome == RecoveryOutcome.Corrupt) + throw new InvalidDataException( + $"Journal recovery failed in {report.DamagedSegment}: {report.Failure}."); - try + if (report.Outcome == RecoveryOutcome.TruncatedTail) { - JournalRecord.Write(buffer, JournalRecordType.SegmentHeader, LastSequence, - DateTime.UtcNow.Ticks, header); - _segment.Write(buffer, 0, size); - _segmentLength += size; + RepairTail(report); + report = JournalReader.Recover(_directory, expectedSessionId: SessionId, + expectedInitialSequence: _initialSequence); + + if (report.Outcome != RecoveryOutcome.Clean) + throw new InvalidDataException("The journal tail could not be repaired."); + + segments = SegmentFiles(_directory); } - finally + + if (!report.HasSequencedRecords && report.NextSequence != _initialSequence) + throw new InvalidDataException("The journal initial sequence does not match."); + + _nextSequence = report.NextSequence; + _hasSequencedRecords = report.HasSequencedRecords; + + if (segments.Count == 0) + { + _segmentIndex = 0; + OpenNewSegment(); + return; + } + + _segmentIndex = IndexOf(segments[^1]); + _segment = OpenSegment(segments[^1], FileMode.Open); + _segmentLength = _segment.Length; + _segment.Position = _segmentLength; + } + + private static void RepairTail(RecoveryReport report) + { + if (report.ValidBytesInDamagedSegment == 0) { - ArrayPool.Shared.Return(buffer); + File.Delete(report.DamagedSegment); + return; } + + using var stream = new FileStream(report.DamagedSegment, FileMode.Open, FileAccess.Write, + FileShare.Read, bufferSize: 1); + stream.SetLength(report.ValidBytesInDamagedSegment); + stream.Flush(flushToDisk: true); + } + + private void OpenNewSegment() + { + var path = Path.Combine(_directory, $"{SegmentPrefix}{_segmentIndex:D9}{SegmentSuffix}"); + _segment = OpenSegment(path, FileMode.CreateNew); + _segmentLength = 0; + + Span payload = stackalloc byte[16]; + BinaryPrimitives.WriteUInt64LittleEndian(payload, SessionId); + BinaryPrimitives.WriteUInt64LittleEndian(payload.Slice(8), _nextSequence); + + var size = JournalRecord.SizeFor(payload.Length); + Span record = stackalloc byte[size]; + JournalRecord.Write(record, JournalRecordType.SegmentHeader, _nextSequence, + DateTime.UtcNow.Ticks, payload); + _segment.Write(record); + _segmentLength = size; + _dirty = true; + } + + private static FileStream OpenSegment(string path, FileMode mode) + => new(path, mode, FileAccess.Write, FileShare.ReadWrite, bufferSize: 1, + FileOptions.SequentialScan); + + private void EnsureWritable() + { + ObjectDisposedException.ThrowIf(_disposed, this); + + if (_failure is not null) + throw new IOException("The journal is fail-stopped after an I/O error.", _failure); } internal static List SegmentFiles(string directory) => Directory.Exists(directory) - ? Directory.GetFiles(directory, SegmentPrefix + "*" + SegmentSuffix).OrderBy(f => f).ToList() + ? Directory.GetFiles(directory, SegmentPrefix + "*" + SegmentSuffix) + .OrderBy(path => path, StringComparer.Ordinal).ToList() : new List(); - private static int IndexOf(string path) + internal static int IndexOf(string path) { var name = Path.GetFileNameWithoutExtension(path); - return int.TryParse(name.AsSpan(SegmentPrefix.Length), out var index) ? index : 0; + return name.StartsWith(SegmentPrefix, StringComparison.Ordinal) && + int.TryParse(name.AsSpan(SegmentPrefix.Length), out var index) + ? index + : throw new InvalidDataException($"Invalid segment name: {path}."); } public void Dispose() { - lock (_gate) - { - if (_disposed) - return; + var stopSync = false; - _disposed = true; - - try + try + { + lock (_gate) { - ForceToDevice(); + if (_disposed) + return; + + stopSync = true; + _syncShutdown?.Set(); + + try + { + if (_failure is null) + ForceToDevice(); + } + finally + { + _disposed = true; + _segment?.Dispose(); + _writerLease.Dispose(); + ActiveWriters.TryRemove(_leaseKey, out _); + } } - finally + } + finally + { + if (stopSync) { - _segment.Dispose(); + _syncThread?.Join(); + _syncShutdown?.Dispose(); } } } diff --git a/Common/Feed/FeedRecoveryCoordinator.cs b/Common/Feed/FeedRecoveryCoordinator.cs new file mode 100644 index 0000000..5dfdab1 --- /dev/null +++ b/Common/Feed/FeedRecoveryCoordinator.cs @@ -0,0 +1,119 @@ +using System; +using System.IO; +using System.Net.Sockets; +using System.Threading; +using System.Threading.Tasks; +using MarketData.Common.Durability; + +namespace MarketData.Common.Feed +{ + public enum GapRecoveryResult : byte + { + NotNeeded, + Repaired, + SnapshotRequired, + InvalidPacket, + } + + /// Serializes live packets with exact unicast gap fill. + public sealed class FeedRecoveryCoordinator + { + private readonly FeedDecoder _decoder; + private readonly RetransmissionClient _client; + private readonly SemaphoreSlim _gate = new(1, 1); + private int _repairing; + + public FeedRecoveryCoordinator(FeedDecoder decoder, RetransmissionClient client) + { + _decoder = decoder ?? throw new ArgumentNullException(nameof(decoder)); + _client = client ?? throw new ArgumentNullException(nameof(client)); + } + + public bool IsRepairing => Volatile.Read(ref _repairing) != 0; + + public async ValueTask ConsumeAsync(ReadOnlyMemory packet, + CancellationToken token = default) + { + if (!FeedProtocol.TryReadHeader(packet.Span, out var header, out _)) + { + _decoder.Consume(packet.Span); + return GapRecoveryResult.InvalidPacket; + } + + await _gate.WaitAsync(token).ConfigureAwait(false); + + try + { + var activeSession = _decoder.SessionId; + var expected = _decoder.ExpectedSequence; + var missing = activeSession == header.SessionId && expected < header.FirstSequence; + + if (missing) + Interlocked.Exchange(ref _repairing, 1); + + _decoder.Consume(packet.Span); + + if (!missing) + return GapRecoveryResult.NotNeeded; + + var response = await _client.RequestDetailedAsync(header.SessionId, expected, + header.FirstSequence - 1, token).ConfigureAwait(false); + + if (response.Status != RetransmissionStatus.Success || + !ApplyExactPrefix(response.Messages, header.SessionId, expected, + header.FirstSequence)) + { + _decoder.FlushGaps(); + return GapRecoveryResult.SnapshotRequired; + } + + return _decoder.HeldPackets == 0 && _decoder.ExpectedSequence >= + header.FirstSequence + header.MessageCount + ? GapRecoveryResult.Repaired + : GapRecoveryResult.SnapshotRequired; + } + catch (OperationCanceledException) when (token.IsCancellationRequested) + { + throw; + } + catch (Exception error) when (error is IOException or SocketException or + OperationCanceledException) + { + _decoder.FlushGaps(); + return GapRecoveryResult.SnapshotRequired; + } + finally + { + Interlocked.Exchange(ref _repairing, 0); + _gate.Release(); + } + } + + public async ValueTask FlushGapsAsync(CancellationToken token = default) + { + await _gate.WaitAsync(token).ConfigureAwait(false); + + try { _decoder.FlushGaps(); } + finally { _gate.Release(); } + } + + private bool ApplyExactPrefix(System.Collections.Generic.List packets, + ulong sessionId, ulong expected, ulong resume) + { + var cursor = expected; + + foreach (var packet in packets) + { + if (!FeedProtocol.TryReadHeader(packet.Payload, out var header, out _) || + header.SessionId != sessionId || header.FirstSequence != cursor || + packet.Sequence != cursor || packet.MessageCount != header.MessageCount) + return false; + + _decoder.Consume(packet.Payload); + cursor = header.FirstSequence + header.MessageCount; + } + + return cursor == resume; + } + } +} diff --git a/Common/Feed/MulticastPublisher.cs b/Common/Feed/MulticastPublisher.cs index 82cfd54..affe938 100644 --- a/Common/Feed/MulticastPublisher.cs +++ b/Common/Feed/MulticastPublisher.cs @@ -6,63 +6,44 @@ using System.Net.Sockets; using System.Security.Cryptography; using System.Threading; +using MarketData.Common.Durability; namespace MarketData.Common.Feed { - /// - /// Publishes the feed as sequenced multicast datagrams. - /// - /// - /// - /// The point of the whole exercise: publishing costs one send regardless of how many - /// subscribers are listening. The unicast path this replaces performs one write per - /// subscriber per update, so its cost - and the latency spread across the subscriber - /// population - grows linearly with the audience. Here the network performs the replication, - /// and the server does not know or care how many receivers exist. - /// - /// - /// That property is bought with reliability. UDP multicast has no retransmission and no - /// backpressure: a subscriber that cannot keep up simply loses packets, and the publisher - /// never finds out. What makes this workable is the sequence number on every packet, which - /// turns silent loss into detectable loss, plus a periodic snapshot that gives a subscriber - /// which has detected a gap a way back to a known-good state. - /// - /// - /// Messages are batched into a packet up to the fragmentation threshold, which amortises the - /// per-datagram cost - syscall, IP and UDP headers - over many updates. Batching trades a - /// little latency for a lot of throughput, so the batch is also flushed on a deadline rather - /// than only when full. - /// - /// + /// Publishes sealed, journal-first feed packets over one or two multicast lines. public sealed class MulticastPublisher : IDisposable { - public ulong Sequence => (ulong)Interlocked.Read(ref _sequence); + public ulong Sequence { get { lock (_lock) return _sequence; } } public ulong SessionId { get; } public long PacketsSent => Interlocked.Read(ref _packetsSent); public long MessagesSent => Interlocked.Read(ref _messagesSent); public long BytesSent => Interlocked.Read(ref _bytesSent); public long SendFailures => Interlocked.Read(ref _sendFailures); + public long JournalFailures => Interlocked.Read(ref _journalFailures); - /// - /// Optional second group carrying an identical copy of the feed. Real exchanges publish an - /// A and a B line over disjoint paths so that a drop on one is covered by the other; - /// subscribers take whichever copy arrives first. It costs one extra send per packet - - /// still independent of the subscriber count - and roughly squares the probability that a - /// given packet is lost to every subscriber. - /// + /// Optional B line carrying the identical sealed packet. public MulticastPublisher(IPAddress group, int port, IPAddress @interface = null, int maxBatch = 64, - IPAddress redundantGroup = null, int redundantPort = 0, ulong sessionId = 0) + IPAddress redundantGroup = null, int redundantPort = 0, ulong sessionId = 0, + WriteAheadJournal journal = null) { ArgumentNullException.ThrowIfNull(group); if ((uint)(port - 1) >= 65535) throw new ArgumentOutOfRangeException(nameof(port)); + if (maxBatch is < 1 or > ushort.MaxValue) + throw new ArgumentOutOfRangeException(nameof(maxBatch)); _endpoint = new IPEndPoint(group, port); _redundantEndpoint = redundantGroup is null ? null : new IPEndPoint(redundantGroup, redundantPort > 0 ? redundantPort : port); - _maxBatch = Math.Max(1, maxBatch); - SessionId = sessionId == 0 ? CreateSessionId() : sessionId; + _maxBatch = maxBatch; + SessionId = sessionId == 0 ? NewSessionId() : sessionId; + + if (journal is not null && journal.SessionId != SessionId) + throw new ArgumentException("Publisher and journal sessions differ.", nameof(journal)); + + _journal = journal; + _sequence = journal?.NextSequence ?? 0; _socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp); _socket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.MulticastTimeToLive, 1); @@ -77,7 +58,7 @@ public MulticastPublisher(IPAddress group, int port, IPAddress @interface = null } } - private static ulong CreateSessionId() + public static ulong NewSessionId() { Span bytes = stackalloc byte[sizeof(ulong)]; @@ -95,6 +76,8 @@ public void Publish(FeedMessageType type, int instrumentId, Side side, PriceLeve { lock (_lock) { + ThrowIfFaulted(); + if (_pending == _maxBatch || _offset + FeedProtocol.IncrementalSize > FeedProtocol.MaxPacketSize) FlushLocked(); @@ -109,6 +92,8 @@ public void PublishSnapshot(int instrumentId, ReadOnlySpan bids, Rea lock (_lock) { + ThrowIfFaulted(); + if (_pending == _maxBatch || _offset + size > FeedProtocol.MaxPacketSize) FlushLocked(); @@ -120,7 +105,10 @@ public void PublishSnapshot(int instrumentId, ReadOnlySpan bids, Rea public void Flush() { lock (_lock) + { + ThrowIfFaulted(); FlushLocked(); + } } private void FlushLocked() @@ -128,26 +116,47 @@ private void FlushLocked() if (_pending == 0) return; - // Stamped at the moment of transmission rather than of generation, because everything - // before this point is measured separately and a subscriber can only observe from here. + // Timestamp the sealed packet at the publication boundary. FeedProtocol.WriteHeader(_buffer.AsSpan(0, _offset), (ushort)_pending, SessionId, - (ulong)_sequence, Stopwatch.GetTimestamp()); + _sequence, Stopwatch.GetTimestamp()); + + if (_journal is not null) + { + try + { + _journal.AppendPacket(_buffer.AsSpan(0, _offset)); + } + catch + { + _faulted = true; + Interlocked.Increment(ref _journalFailures); + throw; + } + } - var successfulSends = TrySend(_endpoint) ? 1 : 0; + var successfulSends = 0; - if (_redundantEndpoint is not null && TrySend(_redundantEndpoint)) - successfulSends++; + try + { + if (TrySend(_endpoint)) + successfulSends++; - // Sequence advances even when every send fails so downstream loss is detectable. - Interlocked.Add(ref _sequence, _pending); - Interlocked.Add(ref _packetsSent, successfulSends); - Interlocked.Add(ref _bytesSent, (long)_offset * successfulSends); + if (_redundantEndpoint is not null && TrySend(_redundantEndpoint)) + successfulSends++; + } + finally + { + // A journalled or partially sent packet cannot reuse its sequence. + _sequence = checked(_sequence + (uint)_pending); + Interlocked.Add(ref _packetsSent, successfulSends); + Interlocked.Add(ref _bytesSent, (long)_offset * successfulSends); - if (successfulSends > 0) - Interlocked.Add(ref _messagesSent, _pending); + if (successfulSends > 0) + Interlocked.Add(ref _messagesSent, _pending); - _pending = 0; - _offset = FeedProtocol.HeaderSize; + _pending = 0; + _offset = FeedProtocol.HeaderSize; + } } private bool TrySend(EndPoint endpoint) @@ -168,23 +177,39 @@ public void Dispose() { lock (_lock) { - FlushLocked(); - _socket.Dispose(); + try + { + if (!_faulted) + FlushLocked(); + } + finally + { + _socket.Dispose(); + } } } + private void ThrowIfFaulted() + { + if (_faulted) + throw new InvalidOperationException("Publisher is fail-stopped after a journal error."); + } + private readonly IPEndPoint _endpoint; private readonly IPEndPoint _redundantEndpoint; private readonly Socket _socket; + private readonly WriteAheadJournal _journal; private readonly int _maxBatch; private readonly object _lock = new object(); private readonly byte[] _buffer = new byte[FeedProtocol.MaxPacketSize]; private int _offset = FeedProtocol.HeaderSize; private int _pending; - private long _sequence; + private ulong _sequence; private long _packetsSent; private long _messagesSent; private long _bytesSent; private long _sendFailures; + private long _journalFailures; + private bool _faulted; } } diff --git a/Common/Feed/MulticastSubscriber.cs b/Common/Feed/MulticastSubscriber.cs index 8f8da90..56acbfa 100644 --- a/Common/Feed/MulticastSubscriber.cs +++ b/Common/Feed/MulticastSubscriber.cs @@ -6,17 +6,11 @@ using System.Net.Sockets; using System.Threading; using System.Threading.Tasks; +using MarketData.Common.Durability; namespace MarketData.Common.Feed { - /// - /// Joins a multicast group and feeds received datagrams to a . - /// - /// - /// Nothing here interprets the feed; this type is only the socket. The interesting behaviour - - /// loss detection, staleness, recovery - lives in the decoder, where it can be tested without - /// a network. - /// + /// Feeds one or two multicast lines through sequencing and optional gap fill. public sealed class MulticastSubscriber : IDisposable { public FeedDecoder Decoder { get; } @@ -24,17 +18,17 @@ public sealed class MulticastSubscriber : IDisposable public MulticastSubscriber(IPAddress group, int port, IPAddress @interface, Func bookFactory, int receiveBufferBytes = 1 << 20, - IPAddress redundantGroup = null, int redundantPort = 0) + IPAddress redundantGroup = null, int redundantPort = 0, + RetransmissionClient retransmission = null) { Decoder = new FeedDecoder(bookFactory); + _recovery = retransmission is null ? null : + new FeedRecoveryCoordinator(Decoder, retransmission); _sockets.Add(Join(group, port, @interface, receiveBufferBytes)); if (redundantGroup is not null) { - // Both lines feed the same decoder. Its duplicate suppression performs the - // arbitration: the first copy of a packet to arrive wins, the second is discarded, - // and a packet dropped on one line leaves no gap so long as the other delivers it. _sockets.Add(Join(redundantGroup, redundantPort > 0 ? redundantPort : port, @interface, receiveBufferBytes)); } @@ -45,8 +39,6 @@ private static Socket Join(IPAddress group, int port, IPAddress @interface, int var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp); socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseAddress, true); - // A generous receive buffer is the subscriber's only shock absorber: a multicast feed - // applies no backpressure, so whatever the socket cannot hold is simply gone. socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReceiveBuffer, receiveBufferBytes); socket.Bind(new IPEndPoint(IPAddress.Any, port)); socket.SetSocketOption(SocketOptionLevel.IP, SocketOptionName.AddMembership, @@ -54,50 +46,19 @@ private static Socket Join(IPAddress group, int port, IPAddress @interface, int return socket; } - /// Receives until is cancelled. - public void Receive(CancellationToken token) + /// Maximum reorder hold time before declaring loss. + public TimeSpan GapTimeout { - var socket = _sockets[0]; - var buffer = new byte[FeedProtocol.MaxPacketSize]; - socket.ReceiveTimeout = 250; - - while (!token.IsCancellationRequested) + get => _gapTimeout; + set { - int received; - - try - { - received = socket.Receive(buffer); - } - catch (SocketException e) when (e.SocketErrorCode == SocketError.TimedOut) - { - continue; - } - catch (Exception) - { - return; - } - - Decoder.Consume(buffer.AsSpan(0, received)); + if (value <= TimeSpan.Zero || value > TimeSpan.FromMinutes(1)) + throw new ArgumentOutOfRangeException(nameof(value)); + _gapTimeout = value; } } - /// - /// Receives asynchronously until is cancelled. - /// - /// - /// A blocking receive costs a dedicated thread per subscriber, which stops being viable at - /// a few hundred subscribers and makes the harness - not the server - the thing under - /// test. This awaits the socket instead, so thousands of subscribers share the thread pool - /// and the measurement stays about the feed. - /// - /// - /// Time out-of-order packets may be held before the hole in front of them is ruled lost. - /// Must exceed the plausible delay between the A and B copies of a packet, or arbitration - /// would be reported as loss. - /// - public TimeSpan GapTimeout { get; set; } = TimeSpan.FromMilliseconds(20); - + /// Receives asynchronously until cancellation. public Task ReceiveAsync(CancellationToken token) => Task.WhenAll(_sockets.Select(socket => ReceiveAsync(socket, token)).Append(RunGapTimerAsync(token))); @@ -123,6 +84,12 @@ private async Task RunGapTimerAsync(CancellationToken token) var expected = Decoder.ExpectedSequence; + if (_recovery?.IsRepairing == true) + { + stalledFor = TimeSpan.Zero; + continue; + } + if (Decoder.HeldPackets == 0 || expected != lastExpected) { lastExpected = expected; @@ -134,7 +101,11 @@ private async Task RunGapTimerAsync(CancellationToken token) if (stalledFor >= GapTimeout) { - Decoder.FlushGaps(); + if (_recovery is null) + Decoder.FlushGaps(); + else + await _recovery.FlushGapsAsync(token).ConfigureAwait(false); + stalledFor = TimeSpan.Zero; lastExpected = Decoder.ExpectedSequence; } @@ -167,7 +138,11 @@ private async Task ReceiveAsync(Socket socket, CancellationToken token) return; } - Decoder.Consume(buffer.AsSpan(0, received)); + if (_recovery is null) + Decoder.Consume(buffer.AsSpan(0, received)); + else + await _recovery.ConsumeAsync(buffer.AsMemory(0, received), token) + .ConfigureAwait(false); } } @@ -178,5 +153,7 @@ public void Dispose() } private readonly List _sockets = new List(2); + private readonly FeedRecoveryCoordinator _recovery; + private TimeSpan _gapTimeout = TimeSpan.FromMilliseconds(20); } } diff --git a/Common/Governance/Compatibility.cs b/Common/Governance/Compatibility.cs index e5cf50e..e8e70bd 100644 --- a/Common/Governance/Compatibility.cs +++ b/Common/Governance/Compatibility.cs @@ -10,15 +10,10 @@ public enum CompatibilityKind /// Identical layout. Identical, - /// - /// A reader of the old version can still read the new one. Achieved by adding optional - /// fields after everything the old reader knows about. - /// + /// Old readers can consume the new layout. BackwardCompatible, - /// - /// A reader of the new version can read the old one, but not the reverse. - /// + /// New readers can consume the old layout. ForwardCompatible, /// Neither direction is safe. Requires a coordinated cutover. @@ -35,35 +30,7 @@ public sealed record CompatibilityReport( Kind is CompatibilityKind.Identical or CompatibilityKind.BackwardCompatible; } - /// - /// Decides mechanically whether a schema change can be rolled out without a cutover. - /// - /// - /// - /// This exists because the question "is this change safe?" is asked at review time, answered - /// from memory, and answered wrong. The rules are simple enough to state and therefore simple - /// enough to check: - /// - /// - /// Removing a message type or a field breaks readers that use it. - /// Moving a field breaks every reader, because offsets are the contract. - /// Changing a field's type or width breaks every reader, for the same reason. - /// Adding a required field breaks old readers, which will not populate it. - /// - /// Adding an optional field beyond the end of the old layout is safe: old readers - /// stop where they always did and never see it. - /// - /// - /// Adding an optional field inside the old layout is not safe, even though it sounds - /// like it should be - it necessarily displaces something. - /// - /// - /// - /// Renaming counts as remove-plus-add, deliberately. A rename is invisible on the wire but - /// changes what code binds to, and treating it as safe is how a field quietly comes to mean - /// something different from what its readers assume. - /// - /// + /// Classifies schema changes and reports field-level breaks. public static class Compatibility { public static CompatibilityReport Compare(Schema older, Schema newer) @@ -71,12 +38,10 @@ public static CompatibilityReport Compare(Schema older, Schema newer) ArgumentNullException.ThrowIfNull(older); ArgumentNullException.ThrowIfNull(newer); - if (older.Fingerprint == newer.Fingerprint) + if (SameLayout(older, newer)) return new CompatibilityReport(CompatibilityKind.Identical, Array.Empty()); var breaks = new List(); - var additionsOnly = true; - foreach (var oldMessage in older.Messages) { var newMessage = newer.Find(oldMessage.TypeCode); @@ -86,7 +51,6 @@ public static CompatibilityReport Compare(Schema older, Schema newer) breaks.Add(new CompatibilityBreak( $"message type {oldMessage.TypeCode} ({oldMessage.Name}) was removed", oldMessage.Name, null)); - additionsOnly = false; continue; } @@ -95,7 +59,6 @@ public static CompatibilityReport Compare(Schema older, Schema newer) breaks.Add(new CompatibilityBreak( $"message type {oldMessage.TypeCode} was renamed from {oldMessage.Name} to {newMessage.Name}", oldMessage.Name, null)); - additionsOnly = false; } var oldSize = oldMessage.Size; @@ -110,7 +73,6 @@ public static CompatibilityReport Compare(Schema older, Schema newer) breaks.Add(new CompatibilityBreak( $"field {oldField.Name} was removed or renamed", oldMessage.Name, oldField.Name)); - additionsOnly = false; continue; } @@ -119,7 +81,6 @@ public static CompatibilityReport Compare(Schema older, Schema newer) breaks.Add(new CompatibilityBreak( $"field {oldField.Name} moved from offset {oldField.Offset} to {newField.Offset}", oldMessage.Name, oldField.Name)); - additionsOnly = false; } if (newField.Type != oldField.Type || newField.Length != oldField.Length) @@ -128,57 +89,79 @@ public static CompatibilityReport Compare(Schema older, Schema newer) $"field {oldField.Name} changed from {oldField.Type}({oldField.Length}) " + $"to {newField.Type}({newField.Length})", oldMessage.Name, oldField.Name)); - additionsOnly = false; + } + + if (newField.Since != oldField.Since || newField.Required != oldField.Required) + { + breaks.Add(new CompatibilityBreak( + $"field {oldField.Name} changed evolution metadata", + oldMessage.Name, oldField.Name)); } } - // Anything new in an existing message must be optional and must sit beyond where - // the old reader stops. foreach (var addedField in newMessage.Fields.Where(f => oldMessage.Fields.All(o => !string.Equals(o.Name, f.Name, StringComparison.Ordinal)))) { - if (addedField.Required) - { - breaks.Add(new CompatibilityBreak( - $"required field {addedField.Name} was added; old writers will not populate it", - newMessage.Name, addedField.Name)); - additionsOnly = false; - } - else if (addedField.Offset < oldSize) - { - breaks.Add(new CompatibilityBreak( - $"optional field {addedField.Name} was added at offset {addedField.Offset}, " + - $"inside the previous layout which ended at {oldSize}", - newMessage.Name, addedField.Name)); - additionsOnly = false; - } + var location = addedField.Offset < oldSize + ? $"inside the previous layout which ended at {oldSize}" + : "to a message without a length-delimited envelope"; + breaks.Add(new CompatibilityBreak( + $"{(addedField.Required ? "required" : "optional")} field " + + $"{addedField.Name} was added {location}", + newMessage.Name, addedField.Name)); } } if (breaks.Count > 0) return new CompatibilityReport(CompatibilityKind.Breaking, breaks); - // No breaks. Whether new message types were added decides the direction: an old reader - // copes with them only by ignoring unknown type codes, which this protocol does. - var addedMessages = newer.Messages - .Any(m => older.Find(m.TypeCode) is null); + var addedMessages = newer.Messages.Where(message => older.Find(message.TypeCode) is null) + .ToArray(); - var kind = additionsOnly - ? CompatibilityKind.BackwardCompatible - : CompatibilityKind.ForwardCompatible; + if (addedMessages.Length > 0) + { + return new CompatibilityReport(CompatibilityKind.ForwardCompatible, + addedMessages.Select(message => new CompatibilityBreak( + $"message type {message.TypeCode} ({message.Name}) was added; " + + "old readers cannot skip unframed message types", + message.Name, null)).ToArray()); + } + + return new CompatibilityReport(CompatibilityKind.Breaking, new[] + { + new CompatibilityBreak("fingerprints differ without a classified layout change", null, null), + }); + } + + private static bool SameLayout(Schema left, Schema right) + { + if (left.Messages.Count != right.Messages.Count) + return false; + + foreach (var leftMessage in left.Messages) + { + var rightMessage = right.Find(leftMessage.TypeCode); + if (rightMessage is null || + !string.Equals(leftMessage.Name, rightMessage.Name, StringComparison.Ordinal) || + leftMessage.Fields.Count != rightMessage.Fields.Count) + return false; + + foreach (var leftField in leftMessage.Fields) + { + var rightField = rightMessage.Fields.FirstOrDefault(field => + string.Equals(field.Name, leftField.Name, StringComparison.Ordinal)); + + if (rightField is null || rightField.Type != leftField.Type || + rightField.Offset != leftField.Offset || rightField.Length != leftField.Length || + rightField.Since != leftField.Since || rightField.Required != leftField.Required) + return false; + } + } - return new CompatibilityReport( - addedMessages ? CompatibilityKind.BackwardCompatible : kind, - Array.Empty()); + return true; } - /// - /// Throws unless can be deployed without a coordinated cutover. - /// - /// - /// Meant to be called from a test, so a breaking change fails the build with the reasons - /// listed rather than being discovered by a subscriber in production. - /// + /// Throws when independent deployment is unsafe. public static void AssertDeployableAgainst(Schema older, Schema newer) { var report = Compare(older, newer); diff --git a/Common/Governance/FeedSchemas.cs b/Common/Governance/FeedSchemas.cs index 155a6d0..7b9e2aa 100644 --- a/Common/Governance/FeedSchemas.cs +++ b/Common/Governance/FeedSchemas.cs @@ -1,66 +1,44 @@ using System; using System.Collections.Generic; using System.Linq; +using MarketData.Common.Feed; namespace MarketData.Common.Governance { - /// - /// The wire contract of the live feed, declared as data so it can be diffed. - /// - /// - /// These layouts mirror FeedProtocol. Keeping them in step is enforced by - /// SchemaGovernanceTests, which compares the declared sizes against the constants the - /// encoder actually uses - a schema that has drifted from its encoder is worse than no schema, - /// because it will be believed. - /// + /// Machine-readable layouts mirrored from FeedProtocol. public static class FeedSchemas { - public const byte IncrementalTypeCode = 1; - public const byte SnapshotHeaderTypeCode = 4; - public const byte HeartbeatTypeCode = 5; + public const byte AddTypeCode = (byte)FeedMessageType.Add; + public const byte ReplaceTypeCode = (byte)FeedMessageType.Replace; + public const byte RemoveTypeCode = (byte)FeedMessageType.Remove; + public const byte SnapshotHeaderTypeCode = (byte)FeedMessageType.Snapshot; + public const byte HeartbeatTypeCode = (byte)FeedMessageType.Heartbeat; /// Version 1: the original layout, retained so compatibility can be tested. public static Schema V1 { get; } = new(1, new[] { - new MessageSchema("Incremental", IncrementalTypeCode, new[] - { - new SchemaField("MessageType", FieldType.UInt8, 0, 1), - new SchemaField("Side", FieldType.UInt8, 1, 1), - new SchemaField("InstrumentId", FieldType.UInt32, 2, 4), - new SchemaField("Price", FieldType.Int32, 6, 4), - new SchemaField("Quantity", FieldType.UInt32, 10, 4), - }), + Incremental("Add", AddTypeCode), + Incremental("Replace", ReplaceTypeCode), + Incremental("Remove", RemoveTypeCode), new MessageSchema("SnapshotHeader", SnapshotHeaderTypeCode, new[] { new SchemaField("MessageType", FieldType.UInt8, 0, 1), - new SchemaField("InstrumentId", FieldType.UInt32, 1, 4), + new SchemaField("InstrumentId", FieldType.Int32, 1, 4), new SchemaField("BidLevels", FieldType.UInt8, 5, 1), new SchemaField("AskLevels", FieldType.UInt8, 6, 1), }), }); - /// - /// Version 2: the current layout, adding a heartbeat message. - /// - /// - /// Adding a whole message type is backward compatible in this protocol because unknown - /// type codes are skipped rather than fatal - which is a property of the decoder, not a - /// law of nature, and is asserted by test. - /// + /// Current layout, including heartbeat. public static Schema V2 { get; } = new(2, new[] { - new MessageSchema("Incremental", IncrementalTypeCode, new[] - { - new SchemaField("MessageType", FieldType.UInt8, 0, 1), - new SchemaField("Side", FieldType.UInt8, 1, 1), - new SchemaField("InstrumentId", FieldType.UInt32, 2, 4), - new SchemaField("Price", FieldType.Int32, 6, 4), - new SchemaField("Quantity", FieldType.UInt32, 10, 4), - }), + Incremental("Add", AddTypeCode), + Incremental("Replace", ReplaceTypeCode), + Incremental("Remove", RemoveTypeCode), new MessageSchema("SnapshotHeader", SnapshotHeaderTypeCode, new[] { new SchemaField("MessageType", FieldType.UInt8, 0, 1), - new SchemaField("InstrumentId", FieldType.UInt32, 1, 4), + new SchemaField("InstrumentId", FieldType.Int32, 1, 4), new SchemaField("BidLevels", FieldType.UInt8, 5, 1), new SchemaField("AskLevels", FieldType.UInt8, 6, 1), }), @@ -72,25 +50,33 @@ public static class FeedSchemas public static Schema Current => V2; - public static IReadOnlyList All { get; } = new[] { V1, V2 }; + public static IReadOnlyList All { get; } = + Array.AsReadOnly(new[] { V1, V2 }); + + private static MessageSchema Incremental(string name, byte typeCode) => new(name, typeCode, + new[] + { + new SchemaField("MessageType", FieldType.UInt8, 0, 1), + new SchemaField("InstrumentId", FieldType.Int32, 1, 4), + new SchemaField("Price", FieldType.Int32, 5, 4), + new SchemaField("Quantity", FieldType.UInt32, 9, 4), + new SchemaField("Side", FieldType.UInt8, 13, 1), + }); } - /// - /// Holds the known schema versions and settles which one a session speaks. - /// - /// - /// Negotiation happens once, at session start, and its result is a single version for the life - /// of the session. Renegotiating mid-stream would mean a subscriber's decoder changing shape - /// while sequenced messages are in flight, which is unresolvable: a packet already on the wire - /// belongs to the old layout and there is no way to say so after the fact. - /// + /// Immutable registry for session-start schema negotiation. public sealed class SchemaRegistry { private readonly Dictionary _byVersion; public SchemaRegistry(IEnumerable schemas) { - _byVersion = schemas.ToDictionary(schema => schema.Version); + ArgumentNullException.ThrowIfNull(schemas); + var materialized = schemas.ToArray(); + if (materialized.Any(schema => schema is null)) + throw new ArgumentException("Schema entries cannot be null.", nameof(schemas)); + + _byVersion = materialized.ToDictionary(schema => schema.Version); if (_byVersion.Count == 0) throw new ArgumentException("A registry needs at least one schema.", nameof(schemas)); @@ -107,46 +93,33 @@ public Schema Get(int version) public bool TryGet(int version, out Schema schema) => _byVersion.TryGetValue(version, out schema); - /// - /// Settles on the highest version both sides know. - /// - /// - /// Fails rather than falling back when there is no overlap. A silent downgrade to a version - /// the publisher no longer intends to speak is how a subscriber ends up quietly consuming a - /// contract nobody is testing. - /// + /// Returns the highest shared version. public Schema Negotiate(IEnumerable peerVersions) { - var shared = peerVersions.Where(_byVersion.ContainsKey).ToList(); + ArgumentNullException.ThrowIfNull(peerVersions); + var peer = peerVersions.Distinct().ToArray(); + var shared = peer.Where(_byVersion.ContainsKey).ToList(); if (shared.Count == 0) { throw new SchemaNegotiationException( $"no shared schema version: this side speaks [{string.Join(", ", Versions)}], " + - $"the peer speaks [{string.Join(", ", peerVersions)}]"); + $"the peer speaks [{string.Join(", ", peer)}]"); } return _byVersion[shared.Max()]; } - /// - /// Confirms the peer's fingerprint matches what this side believes that version looks like. - /// - /// - /// The check that catches the genuinely dangerous case: two builds that agree they speak - /// "version 2" while disagreeing about what version 2 is, because one of them shipped a - /// layout edit without bumping the number. The version is what they agree on; the - /// fingerprint is what makes the agreement mean something. - /// - public Schema Confirm(int version, ulong peerFingerprint) + /// Validates a peer's version fingerprint. + public Schema Confirm(int version, UInt128 peerFingerprint) { var schema = Get(version); if (schema.Fingerprint != peerFingerprint) { throw new SchemaNegotiationException( - $"schema v{version} fingerprint mismatch: peer sent {peerFingerprint:X16}, " + - $"this build has {schema.Fingerprint:X16}. Same version number, different layout."); + $"schema v{version} fingerprint mismatch: peer sent {peerFingerprint:X32}, " + + $"this build has {schema.Fingerprint:X32}. Same version number, different layout."); } return schema; diff --git a/Common/Governance/Schema.cs b/Common/Governance/Schema.cs index 9f56694..2163953 100644 --- a/Common/Governance/Schema.cs +++ b/Common/Governance/Schema.cs @@ -1,6 +1,9 @@ using System; +using System.Buffers; +using System.Buffers.Binary; using System.Collections.Generic; using System.Linq; +using System.Security.Cryptography; using System.Text; namespace MarketData.Common.Governance @@ -20,15 +23,12 @@ public enum FieldType : byte } /// One field in a versioned message layout. - /// Stable identity. Renaming a field is a breaking change; see below. + /// Stable field identity. /// Wire type. /// Byte offset from the start of the message body. /// Bytes occupied. Fixed by except for ASCII. /// Schema version that introduced the field. - /// - /// Whether a reader must understand the field. A required field added to an existing message is - /// a breaking change; an optional one is not. - /// + /// Whether readers require the field. public sealed record SchemaField( string Name, FieldType Type, @@ -49,92 +49,105 @@ public sealed record SchemaField( _ => throw new ArgumentOutOfRangeException(nameof(type), type, null), }; - public int End => Offset + Length; + public int End => checked(Offset + Length); } /// The layout of one message type at one schema version. public sealed record MessageSchema(string Name, byte TypeCode, IReadOnlyList Fields) { /// Bytes the message occupies, taken from the furthest field. - public int Size => Fields.Count == 0 ? 0 : Fields.Max(field => field.End); + public int Size => Fields.Count == 0 ? 0 : Fields.Max(schemaField => schemaField.End); /// Fields a reader at is expected to know. public IEnumerable FieldsAsOf(int version) => Fields.Where(field => field.Since <= version); } - /// - /// A complete, versioned wire contract. - /// - /// - /// - /// The point of writing the layout down as data rather than leaving it implicit in encoder and - /// decoder source is that compatibility becomes a property that can be checked instead - /// of a claim someone makes in a pull request. Two schema versions can be diffed mechanically, - /// and the rules below decide whether the change is safe. - /// - /// - /// The identity carried on the wire is , not the version number. - /// Version numbers are administrative and can be bumped without changing anything, or - far - /// worse - changed without being bumped. A fingerprint over the actual layout cannot be wrong - /// about what the sender is sending. - /// - /// + /// Immutable versioned wire layout with a deterministic fingerprint. public sealed class Schema { public Schema(int version, IReadOnlyList messages) { if (version < 1) throw new ArgumentOutOfRangeException(nameof(version), version, "Versions start at 1."); + ArgumentNullException.ThrowIfNull(messages); + if (messages.Count == 0) + throw new ArgumentException("A schema needs at least one message.", nameof(messages)); Version = version; - Messages = messages; + var copy = new MessageSchema[messages.Count]; - var duplicateCodes = messages.GroupBy(m => m.TypeCode).FirstOrDefault(g => g.Count() > 1); + for (var i = 0; i < messages.Count; i++) + { + var message = messages[i] ?? + throw new ArgumentException("Message entries cannot be null.", nameof(messages)); + if (message.Fields is null) + throw new ArgumentException($"{message.Name} has no field collection.", nameof(messages)); + + copy[i] = new MessageSchema(message.Name, message.TypeCode, + Array.AsReadOnly(message.Fields.ToArray())); + } + + Messages = Array.AsReadOnly(copy); + + var duplicateCodes = Messages.GroupBy(m => m.TypeCode).FirstOrDefault(g => g.Count() > 1); if (duplicateCodes is not null) throw new ArgumentException($"Type code {duplicateCodes.Key} is used by more than one message.", nameof(messages)); + if (Messages.Select(message => message.Name).Distinct(StringComparer.Ordinal).Count() != + Messages.Count) + throw new ArgumentException("Message names must be unique.", nameof(messages)); - foreach (var message in messages) - Validate(message); + foreach (var message in Messages) + Validate(message, version); - Fingerprint = ComputeFingerprint(messages); + Fingerprint = ComputeFingerprint(Messages); } public int Version { get; } public IReadOnlyList Messages { get; } - /// - /// A stable hash of the layout itself, carried on the wire and compared at session start. - /// - public ulong Fingerprint { get; } + /// Stable layout identity for compatibility and session negotiation. + public UInt128 Fingerprint { get; } public MessageSchema Find(byte typeCode) => Messages.FirstOrDefault(message => message.TypeCode == typeCode); - /// - /// Rejects layouts that are internally impossible, before anything encodes against them. - /// - /// - /// Overlapping fields are the interesting case. They are trivially easy to introduce by - /// hand-editing an offset and essentially undetectable afterwards: the encoder writes both - /// fields, the second silently truncates the first, and the corruption looks like bad data - /// rather than a bad schema. - /// - private static void Validate(MessageSchema message) + private static void Validate(MessageSchema message, int version) { + if (string.IsNullOrWhiteSpace(message.Name) || message.TypeCode == 0 || + message.Fields.Count == 0) + throw new ArgumentException("Messages require a name, type code, and fields."); + if (message.Fields.Any(field => field is null)) + throw new ArgumentException($"{message.Name} contains a null field."); + var ordered = message.Fields.OrderBy(field => field.Offset).ToList(); for (var i = 0; i < ordered.Count; i++) { var field = ordered[i]; + if (string.IsNullOrWhiteSpace(field.Name)) + throw new ArgumentException($"{message.Name} contains an unnamed field."); + if (field.Offset < 0) throw new ArgumentException($"{message.Name}.{field.Name} has a negative offset."); if (field.Length <= 0) throw new ArgumentException($"{message.Name}.{field.Name} has a non-positive length."); + if (field.Since < 1 || field.Since > version) + throw new ArgumentException($"{message.Name}.{field.Name} has an invalid version."); + + if (!Enum.IsDefined(field.Type)) + throw new ArgumentException($"{message.Name}.{field.Name} has an invalid type."); + + try { _ = field.End; } + catch (OverflowException) + { + throw new ArgumentException($"{message.Name}.{field.Name} exceeds the layout bound."); + } + if (field.Type != FieldType.Ascii && field.Length != SchemaField.WidthOf(field.Type)) throw new ArgumentException( $"{message.Name}.{field.Name} is {field.Type} but declares {field.Length} bytes."); @@ -145,47 +158,58 @@ private static void Validate(MessageSchema message) $"{ordered[i - 1].Name} which ends at {ordered[i - 1].End}."); } - if (message.Fields.Select(field => field.Name).Distinct().Count() != message.Fields.Count) + if (message.Fields.Select(field => field.Name).Distinct(StringComparer.Ordinal).Count() != + message.Fields.Count) throw new ArgumentException($"{message.Name} has duplicate field names."); } - /// - /// FNV-1a over the layout. Deterministic across runs and processes, which a managed - /// string hash is explicitly not. - /// - private static ulong ComputeFingerprint(IReadOnlyList messages) + /// Truncated SHA-256 over a length-delimited binary canonical form. + private static UInt128 ComputeFingerprint(IReadOnlyList messages) { - const ulong offsetBasis = 14695981039346656037; - const ulong prime = 1099511628211; + var canonical = new ArrayBufferWriter(); - var hash = offsetBasis; + void WriteByte(byte value) + { + canonical.GetSpan(1)[0] = value; + canonical.Advance(1); + } - void Mix(string text) + void WriteInt32(int value) { - foreach (var b in Encoding.UTF8.GetBytes(text)) - { - hash ^= b; - hash *= prime; - } + BinaryPrimitives.WriteInt32LittleEndian(canonical.GetSpan(sizeof(int)), value); + canonical.Advance(sizeof(int)); } - // Ordered so the fingerprint depends on the layout and not on declaration order. + void WriteString(string text) + { + var length = Encoding.UTF8.GetByteCount(text); + WriteInt32(length); + Encoding.UTF8.GetBytes(text, canonical.GetSpan(length)); + canonical.Advance(length); + } + + WriteInt32(messages.Count); + foreach (var message in messages.OrderBy(m => m.TypeCode)) { - Mix(message.Name); - Mix(message.TypeCode.ToString()); + WriteString(message.Name); + WriteByte(message.TypeCode); + WriteInt32(message.Fields.Count); foreach (var field in message.Fields.OrderBy(f => f.Offset)) { - Mix(field.Name); - Mix(((byte)field.Type).ToString()); - Mix(field.Offset.ToString()); - Mix(field.Length.ToString()); - Mix(field.Required ? "R" : "O"); + WriteString(field.Name); + WriteByte((byte)field.Type); + WriteInt32(field.Offset); + WriteInt32(field.Length); + WriteInt32(field.Since); + WriteByte(field.Required ? (byte)1 : (byte)0); } } - return hash; + Span digest = stackalloc byte[SHA256.HashSizeInBytes]; + SHA256.HashData(canonical.WrittenSpan, digest); + return BinaryPrimitives.ReadUInt128LittleEndian(digest); } } } diff --git a/Common/Reference/InstrumentMaster.cs b/Common/Reference/InstrumentMaster.cs index d91acf3..f60d967 100644 --- a/Common/Reference/InstrumentMaster.cs +++ b/Common/Reference/InstrumentMaster.cs @@ -16,17 +16,10 @@ public enum ReferenceChangeReason Correction, } - /// - /// An instrument's attributes over one half-open interval of time. - /// + /// Instrument attributes over a half-open effective interval. /// Inclusive start. - /// - /// Exclusive end, or while current. - /// - /// - /// When this fact was learned, which is not when it took effect. Both are kept; see - /// . - /// + /// Exclusive end; means current. + /// System time when the fact became known. public sealed record InstrumentRecord( int InstrumentId, string Symbol, @@ -41,112 +34,68 @@ public sealed record InstrumentRecord( public bool CoversAt(DateTime instant) => instant >= EffectiveFrom && instant < EffectiveTo; } - /// - /// Effective-dated instrument reference data with point-in-time lookup. - /// - /// - /// - /// Reference data is not a dictionary of current values, and treating it as one is the single - /// most common way historical analysis goes quietly wrong. A symbol that was reused, a tick - /// size that changed, a split that repriced everything - each means the correct answer to - /// "what were this instrument's attributes?" depends on when you are asking about. A - /// mutable map silently answers every historical question with today's facts. - /// - /// - /// So records are intervals, never overwritten, and lookups take an instant. Amending a fact - /// closes the old interval rather than editing it. - /// - /// - /// Two time axes, deliberately. EffectiveFrom is when a fact became true in the - /// world; RecordedAt is when this system learned it. They differ whenever a correction - /// arrives late, and keeping both is what makes it possible to reproduce a decision made with - /// the information available at the time - which is what an audit actually asks for. Asking - /// only "what was true?" is the wrong question when the dispute is about what was knowable. - /// - /// + /// Single-writer bitemporal instrument reference data. public sealed class InstrumentMaster { private readonly Dictionary> _byInstrument = new(); - /// Adds a record, closing any open interval it supersedes. + /// Appends an immutable reference-data observation. public void Amend(InstrumentRecord record) { ArgumentNullException.ThrowIfNull(record); - if (record.EffectiveTo <= record.EffectiveFrom) - throw new ArgumentException("A record must cover a non-empty interval.", nameof(record)); + if (record.InstrumentId <= 0 || string.IsNullOrWhiteSpace(record.Symbol) || + record.TickSize <= 0 || record.LotSize <= 0 || + string.IsNullOrWhiteSpace(record.Currency) || + record.EffectiveTo <= record.EffectiveFrom || !Enum.IsDefined(record.Reason)) + throw new ArgumentException("Instrument record is invalid.", nameof(record)); if (!_byInstrument.TryGetValue(record.InstrumentId, out var history)) _byInstrument[record.InstrumentId] = history = new List(); - for (var i = 0; i < history.Count; i++) + if (history.Any(existing => existing.RecordedAt == record.RecordedAt && + existing.EffectiveFrom == record.EffectiveFrom)) { - var existing = history[i]; - - // An open interval that the new record starts inside gets closed at that point, - // rather than replaced: the old fact was true for the time it covered, and - // rewriting it would destroy the ability to reproduce past decisions. - if (existing.EffectiveFrom < record.EffectiveFrom && existing.EffectiveTo > record.EffectiveFrom) - history[i] = existing with { EffectiveTo = record.EffectiveFrom }; + throw new InvalidOperationException( + "Reference observations require a unique recorded/effective timestamp pair."); } history.Add(record); - history.Sort((left, right) => left.EffectiveFrom.CompareTo(right.EffectiveFrom)); } /// The record in force at , or null. public InstrumentRecord AsOf(int instrumentId, DateTime instant) => _byInstrument.TryGetValue(instrumentId, out var history) - ? history.LastOrDefault(record => record.CoversAt(instant)) + ? Materialize(history, DateTime.MaxValue) + .LastOrDefault(record => record.CoversAt(instant)) : null; - /// - /// The record in force at as it was known at - /// . - /// - /// - /// The bitemporal query. Answers "what did we believe about this instrument at the time we - /// acted?", which is the question an audit or a reconciliation actually poses, and which a - /// single-axis lookup cannot express at all. - /// + /// Returns effective state using only facts known by . public InstrumentRecord AsKnownAt(int instrumentId, DateTime instant, DateTime asKnownAt) => _byInstrument.TryGetValue(instrumentId, out var history) - ? history - .Where(record => record.CoversAt(instant) && record.RecordedAt <= asKnownAt) - .OrderByDescending(record => record.RecordedAt) - .FirstOrDefault() + ? Materialize(history, asKnownAt).LastOrDefault(record => record.CoversAt(instant)) : null; - /// - /// Resolves a symbol at a point in time. - /// - /// - /// Deliberately not a reverse dictionary. Symbols are recycled - a ticker freed by a - /// delisting is reassigned to an unrelated company - so a symbol alone does not identify an - /// instrument, and a lookup that pretends otherwise will confidently return the wrong one. - /// + /// Resolves every instrument holding a symbol at an effective instant. public IReadOnlyList ResolveSymbol(string symbol, DateTime instant) - => _byInstrument.Values - .SelectMany(history => history) - .Where(record => record.CoversAt(instant) - && string.Equals(record.Symbol, symbol, StringComparison.OrdinalIgnoreCase)) - .ToList(); + { + ArgumentException.ThrowIfNullOrWhiteSpace(symbol); + return _byInstrument.Values + .SelectMany(history => Materialize(history, DateTime.MaxValue)) + .Where(record => record.CoversAt(instant) && + string.Equals(record.Symbol, symbol, StringComparison.OrdinalIgnoreCase)) + .OrderBy(record => record.InstrumentId) + .ToList().AsReadOnly(); + } public IReadOnlyList History(int instrumentId) => _byInstrument.TryGetValue(instrumentId, out var history) - ? history.ToList() + ? Materialize(history, DateTime.MaxValue).AsReadOnly() : Array.Empty(); - public IEnumerable Instruments => _byInstrument.Keys; + public IEnumerable Instruments => _byInstrument.Keys.OrderBy(id => id); - /// - /// Checks that an instrument's history has no gaps or overlaps. - /// - /// - /// A gap means some instant has no answer; an overlap means it has two. Both are silent - /// failures at lookup time - the first returns null and the second returns whichever record - /// happened to sort last - so the structure is validated directly instead. - /// + /// Reports gaps and overlaps in the current effective timeline. public IReadOnlyList Validate(int instrumentId) { var problems = new List(); @@ -157,7 +106,7 @@ public IReadOnlyList Validate(int instrumentId) return problems; } - var ordered = history.OrderBy(record => record.EffectiveFrom).ToList(); + var ordered = Materialize(history, DateTime.MaxValue); for (var i = 1; i < ordered.Count; i++) { @@ -179,5 +128,43 @@ public IReadOnlyList Validate(int instrumentId) return problems; } + + private static List Materialize(List observations, + DateTime asKnownAt) + { + var timeline = new List(observations.Count); + + foreach (var observation in observations + .Where(record => record.RecordedAt <= asKnownAt) + .OrderBy(record => record.RecordedAt) + .ThenBy(record => record.EffectiveFrom)) + { + var record = observation; + + for (var i = timeline.Count - 1; i >= 0; i--) + { + var existing = timeline[i]; + + if (existing.EffectiveFrom == record.EffectiveFrom) + { + timeline.RemoveAt(i); + } + else if (existing.EffectiveFrom < record.EffectiveFrom && + existing.EffectiveTo > record.EffectiveFrom) + { + timeline[i] = existing with { EffectiveTo = record.EffectiveFrom }; + } + } + + timeline.Add(record); + } + + timeline.Sort((left, right) => + { + var effective = left.EffectiveFrom.CompareTo(right.EffectiveFrom); + return effective != 0 ? effective : left.RecordedAt.CompareTo(right.RecordedAt); + }); + return timeline; + } } } diff --git a/Common/Reference/SessionCalendar.cs b/Common/Reference/SessionCalendar.cs index a8c1c52..f36310a 100644 --- a/Common/Reference/SessionCalendar.cs +++ b/Common/Reference/SessionCalendar.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Frozen; using System.Collections.Generic; using System.Linq; @@ -42,31 +43,11 @@ public sealed record TradingHalt(int InstrumentId, DateTime From, DateTime To, s public bool CoversAt(DateTime instant) => instant >= From && instant < To; } - /// - /// When a venue trades, and what that implies about the data it emits. - /// - /// - /// - /// The calendar is not decoration. Most of the ways market data analysis goes wrong are - /// calendar errors wearing a different hat: a "zero return" that was really a holiday, a - /// spread that looks impossibly wide because it was sampled during an auction, a book that - /// appears frozen because the instrument was halted. Each of those is a real number that means - /// something other than what it appears to mean, and only the calendar can say so. - /// - /// - /// So the state is a first-class input, and exists to be - /// asked before a quote is treated as a quote. - /// - /// - /// Times of day are venue-local and holidays are venue-local dates, because that is how venues - /// actually define them. Converting to UTC first would need a time zone and would fold the - /// daylight-saving question into the calendar, where it does not belong. - /// - /// + /// Venue-local session state with holidays, special days, and instrument halts. public sealed class SessionCalendar { - private readonly List _windows; - private readonly HashSet _holidays; + private readonly IReadOnlyList _windows; + private readonly FrozenSet _holidays; private readonly Dictionary> _specialDays; private readonly List _halts = new(); @@ -77,25 +58,41 @@ public SessionCalendar( IEnumerable tradingDays = null, IDictionary> specialDays = null) { + ArgumentException.ThrowIfNullOrWhiteSpace(venue); + ArgumentNullException.ThrowIfNull(windows); + Venue = venue; - _windows = windows.OrderBy(window => window.Start).ToList(); - _holidays = new HashSet((holidays ?? Array.Empty()).Select(d => d.Date)); + var suppliedWindows = windows.ToArray(); + if (suppliedWindows.Any(window => window is null)) + throw new ArgumentException("Session windows cannot be null.", nameof(windows)); + var orderedWindows = suppliedWindows.OrderBy(window => window.Start).ToArray(); + ValidateWindows(orderedWindows, nameof(windows)); + _windows = Array.AsReadOnly(orderedWindows); + _holidays = (holidays ?? Array.Empty()).Select(date => date.Date).ToFrozenSet(); _specialDays = specialDays is null ? new Dictionary>() - : specialDays.ToDictionary(entry => entry.Key.Date, entry => entry.Value); - - TradingDays = new HashSet(tradingDays ?? new[] + : specialDays.ToDictionary(entry => entry.Key.Date, entry => + { + if (entry.Value is null) + throw new ArgumentException("Special-day windows cannot be null.", + nameof(specialDays)); + var supplied = entry.Value.ToArray(); + if (supplied.Any(window => window is null)) + throw new ArgumentException("Special-day windows cannot be null.", + nameof(specialDays)); + var copy = supplied.OrderBy(window => window.Start).ToArray(); + ValidateWindows(copy, nameof(specialDays)); + return (IReadOnlyList)Array.AsReadOnly(copy); + }); + + TradingDays = (tradingDays ?? new[] { DayOfWeek.Monday, DayOfWeek.Tuesday, DayOfWeek.Wednesday, DayOfWeek.Thursday, DayOfWeek.Friday, - }); + }).ToFrozenSet(); - for (var i = 1; i < _windows.Count; i++) - { - if (_windows[i - 1].End > _windows[i].Start) - throw new ArgumentException( - $"{venue}: session windows overlap at {_windows[i].Start}.", nameof(windows)); - } + if (TradingDays.Count == 0 || TradingDays.Any(day => !Enum.IsDefined(day))) + throw new ArgumentException("Trading days are invalid.", nameof(tradingDays)); } public string Venue { get; } @@ -115,7 +112,14 @@ public SessionCalendar( }, holidays); - public void AddHalt(TradingHalt halt) => _halts.Add(halt); + public void AddHalt(TradingHalt halt) + { + ArgumentNullException.ThrowIfNull(halt); + if (halt.InstrumentId <= 0 || halt.To <= halt.From || + string.IsNullOrWhiteSpace(halt.Reason)) + throw new ArgumentException("Trading halt is invalid.", nameof(halt)); + _halts.Add(halt); + } public bool IsTradingDay(DateTime date) => TradingDays.Contains(date.DayOfWeek) && !_holidays.Contains(date.Date); @@ -123,10 +127,6 @@ public bool IsTradingDay(DateTime date) /// The venue's state for an instrument at a venue-local instant. public SessionState StateAt(DateTime localInstant, int instrumentId = 0) { - // A halt overrides the schedule: the clock says continuous, the venue says no. - if (_halts.Any(halt => halt.InstrumentId == instrumentId && halt.CoversAt(localInstant))) - return SessionState.Halted; - if (!IsTradingDay(localInstant)) return SessionState.Closed; @@ -136,21 +136,19 @@ public SessionState StateAt(DateTime localInstant, int instrumentId = 0) foreach (var window in windows) { if (window.Contains(timeOfDay)) - return window.State; + { + return window.State != SessionState.Closed && + _halts.Any(halt => halt.InstrumentId == instrumentId && + halt.CoversAt(localInstant)) + ? SessionState.Halted + : window.State; + } } return SessionState.Closed; } - /// - /// Whether a two-sided quote at this instant means what a quote normally means. - /// - /// - /// The question worth asking before computing a spread, a mid, or a return. During an - /// auction the book is indicative and crossed by design; during a halt it is stale; when - /// closed it is empty. All three produce numbers, and all three of those numbers are - /// misleading. - /// + /// Whether the venue is in continuous trading. public bool IsContinuousTrading(DateTime localInstant, int instrumentId = 0) => StateAt(localInstant, instrumentId) == SessionState.Continuous; @@ -160,8 +158,6 @@ public DateTime NextTransition(DateTime localInstant, int instrumentId = 0) var current = StateAt(localInstant, instrumentId); var probe = localInstant; - // Bounded scan: a venue that does not change state within a fortnight is misconfigured, - // and an unbounded loop here would hang rather than say so. var limit = localInstant.AddDays(14); while (probe < limit) @@ -203,5 +199,21 @@ private IEnumerable CandidateBoundaries(DateTime instant, int instrume yield return date.AddDays(1); } + + private static void ValidateWindows(IReadOnlyList windows, string parameter) + { + if (windows.Count == 0) + throw new ArgumentException("At least one session window is required.", parameter); + + for (var i = 0; i < windows.Count; i++) + { + var window = windows[i]; + if (window is null || !Enum.IsDefined(window.State) || window.Start < TimeSpan.Zero || + window.End > TimeSpan.FromDays(1) || window.End <= window.Start) + throw new ArgumentException("Session window is invalid.", parameter); + if (i > 0 && windows[i - 1].End > window.Start) + throw new ArgumentException("Session windows overlap.", parameter); + } + } } } diff --git a/Common/Server/MulticastOrderbookService.cs b/Common/Server/MulticastOrderbookService.cs index e98e42f..ef1935e 100644 --- a/Common/Server/MulticastOrderbookService.cs +++ b/Common/Server/MulticastOrderbookService.cs @@ -5,38 +5,30 @@ using System.Net; using System.Threading; using System.Threading.Tasks; +using MarketData.Common.Durability; namespace MarketData.Common.Server { - /// - /// Disseminates the feed over multicast instead of per-subscriber unicast streams. - /// - /// - /// - /// Structurally the whole point: performs one encode and - /// at most one send, and contains no reference to subscribers at all. The unicast - /// implementation it replaces walks the subscriber table on every update, so its per-update - /// cost - and the latency spread across that population - grows with the number of listeners. - /// Here the server genuinely does not know how many there are. - /// - /// - /// Because there is no per-subscriber state, there is also no per-subscriber queue, no slow - /// consumer to detect and no backpressure to apply. A subscriber that falls behind loses - /// packets and is responsible for noticing; the periodic snapshot below is what lets it - /// recover. - /// - /// + /// Subscriber-independent multicast dissemination with optional durable gap fill. public sealed class MulticastOrderbookService : IOrderbookService { public MulticastOrderbookService(IPAddress group, int port, IPAddress @interface, int maxBatch, TimeSpan flushInterval, TimeSpan snapshotInterval, IOrderbookManager manager, - IPAddress redundantGroup = null, int redundantPort = 0) + IPAddress redundantGroup = null, int redundantPort = 0, + WriteAheadJournal journal = null, int retransmissionPort = 0) { _publisher = new MulticastPublisher(group, port, @interface, maxBatch, - redundantGroup, redundantPort); + redundantGroup, redundantPort, journal?.SessionId ?? 0, journal); _flushInterval = flushInterval; _snapshotInterval = snapshotInterval; _manager = manager; + _journal = journal; + + if (retransmissionPort != 0 && journal is null) + throw new ArgumentException("Retransmission requires a journal.", nameof(retransmissionPort)); + if (journal is not null && retransmissionPort > 0) + _retransmission = new RetransmissionService(journalDirectory: JournalDirectory(journal), + port: retransmissionPort, address: @interface); } public Task StartAsync() @@ -44,6 +36,7 @@ public Task StartAsync() if (_pump is not null) return Task.CompletedTask; + _retransmission?.Start(); _pump = PumpAsync(_shutdown.Token); return Task.CompletedTask; } @@ -63,8 +56,7 @@ public ValueTask OnOrderbookUpdateAsync(OrderbookUpdate update) new PriceLevel(incremental.Level.Price, incremental.Level.Quantity)); } - // With batching disabled the packet leaves immediately, which is the configuration the - // latency comparison against unicast is run in. + // Zero flush interval disables batching latency. if (_flushInterval <= TimeSpan.Zero) _publisher.Flush(); @@ -95,15 +87,7 @@ private static PriceLevel[] Rent(ref PriceLevel[] buffer, int required) return buffer; } - /// - /// Flushes partial batches on a deadline and republishes full books periodically. - /// - /// - /// The recurring snapshot is the entire recovery story on an unreliable transport. A - /// subscriber that detects a gap has no way to request a retransmission, so its only route - /// back to a correct book is to wait for the next complete one; the interval therefore - /// bounds how long a gapped subscriber stays dark. - /// + /// Flushes partial batches and republishes recovery snapshots. private async Task PumpAsync(CancellationToken token) { var lastSnapshot = Stopwatch.GetTimestamp(); @@ -168,7 +152,7 @@ public OrderbookServiceStatistics GetStatistics() DisseminatedUpdates: _publisher.MessagesSent, SentMessages: _publisher.PacketsSent, DroppedUpdates: 0, - FailedSends: _publisher.SendFailures, + FailedSends: _publisher.SendFailures + _publisher.JournalFailures, OutboundQueued: 0, MaxOutboundQueued: 0); @@ -189,10 +173,22 @@ public void Dispose() // Shutting down; the pump's cancellation is expected. } - _publisher.Dispose(); - _shutdown.Dispose(); + _retransmission?.Dispose(); + + try + { + _publisher.Dispose(); + } + finally + { + _journal?.Dispose(); + _shutdown.Dispose(); + } } + private static string JournalDirectory(WriteAheadJournal journal) + => journal.DirectoryPath; + private static FeedMessageType ToMessageType(OrderbookUpdateType type) => type switch { OrderbookUpdateType.Add => FeedMessageType.Add, @@ -205,6 +201,8 @@ public void Dispose() private readonly TimeSpan _flushInterval; private readonly TimeSpan _snapshotInterval; private readonly IOrderbookManager _manager; + private readonly WriteAheadJournal _journal; + private readonly RetransmissionService _retransmission; private readonly CancellationTokenSource _shutdown = new CancellationTokenSource(); private PriceLevel[] _bidScratch = new PriceLevel[64]; private PriceLevel[] _askScratch = new PriceLevel[64]; diff --git a/Common/Server/OrderbookService.cs b/Common/Server/OrderbookService.cs index 54dc930..c815189 100644 --- a/Common/Server/OrderbookService.cs +++ b/Common/Server/OrderbookService.cs @@ -82,7 +82,7 @@ public OrderbookService(int port, IOrderbookManager orderbookManager, bool verbo : ProcessRingUpdatesAsync(new WeakReference(this), _ringQueue, _ringShutdown.Token); } - private static async Task ProcessIncrementalUpdatesAsync(WeakReference model, + private static async Task ProcessIncrementalUpdatesAsync(WeakReference model, ChannelReader reader, TaskCompletionSource shutdownSource) { @@ -270,12 +270,14 @@ public OrderbookServiceStatistics GetStatistics() var snapshot = _clientSnapshot; var clients = snapshot.Length; long liveDrops = 0; + long liveFailures = 0; long outboundQueued = 0; var maxOutboundQueued = 0; for (var i = 0; i < snapshot.Length; i++) { liveDrops += snapshot[i].DroppedUpdates; + liveFailures += snapshot[i].FailedSends; var queued = snapshot[i].QueuedOutbound; outboundQueued += queued; @@ -292,7 +294,7 @@ public OrderbookServiceStatistics GetStatistics() Interlocked.Read(ref _disseminatedUpdates), Interlocked.Read(ref _sentMessages), Interlocked.Read(ref _droppedUpdates) + liveDrops, - Interlocked.Read(ref _failedSends), + Interlocked.Read(ref _failedSends) + liveFailures, outboundQueued, maxOutboundQueued); } @@ -374,6 +376,7 @@ public override async Task StreamOrderbookUpdates(IAsyncStreamReaderUpdates discarded because this subscriber could not keep up. public long DroppedUpdates => Interlocked.Read(ref _droppedUpdates); + /// Stream writes that failed before teardown. + public long FailedSends => Interlocked.Read(ref _failedSends); + public IReadOnlySet Ids => _subscribedIds; public ServerClient(string host, IServerStreamWriter stream, int queueCapacity) @@ -169,7 +172,7 @@ public async Task PumpAsync(CancellationToken token) } catch (Exception) { - // The subscriber's stream is gone; the call teardown path removes it. + Interlocked.Increment(ref _failedSends); } } @@ -182,6 +185,7 @@ public void Complete() private static long _nextId; private static readonly HashSet _empty = new HashSet(); private long _droppedUpdates; + private long _failedSends; private volatile bool _completed; private readonly IServerStreamWriter _stream = null; private readonly Channel _outbound = null; diff --git a/README.md b/README.md index e8d87e8..c2a1595 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,9 @@ This is a research system, not a production venue. The implemented boundary is e | Public depth | Derived from matching events; no second source of truth | | Dissemination | Bounded gRPC fan-out or sequenced UDP multicast with A/B arbitration | | Wire protocol | Fixed little-endian v2 packets, session identity, exact length, CRC-32C | -| Recovery | Bounded reorder, per-instrument stale generations, atomic snapshots | +| Durability | Single-writer segmented WAL, restart repair, CRC-checked checkpoints | +| Recovery | Bounded reorder, exact TCP gap fill, sparse range index, atomic snapshots | +| Governance | 128-bit layout fingerprints, conservative compatibility, bitemporal reference data | | Validation | Differential/property tests and LOBSTER-derived NASDAQ replay | | Analytics | Allocation-free order-flow imbalance, online regression, stylized facts | @@ -55,12 +57,30 @@ Decoder invariants: - Conflicting A/B packets at the same session and sequence are counted as line divergence. - Exact packet length, message boundaries, sides, quantities, snapshot order, and checksum are validated before state or sequence advances. -- Out-of-order packets are held within a fixed bound. `GapDetected` is the extension point for a - retransmission service; periodic snapshots are the implemented recovery path. +- Out-of-order packets are bounded. Missing ranges use session-scoped TCP gap fill; unavailable or + oversized ranges require a snapshot. The reconstructed book is either current or explicitly stale. It is never silently accepted after modeled loss, corruption, late join, or restart. +## Durability contract + +The publisher appends each sealed packet before either multicast send. One writer owns a journal; +message sequences are contiguous and session-bound. Startup truncates only an incomplete final +record. Bad CRCs, wrong sessions, sequence holes, and missing middle segments fail closed. + +| Policy | Append acknowledgement | +|---|---| +| `OsBuffered` | bytes reached the OS page cache; process crash safe, power loss unsafe | +| `SyncPeriodic` | OS page cache; a dedicated thread runs `fsync` every `SyncInterval` while dirty | +| `SyncEachRecord` | each returned append passed `FileStream.Flush(true)` | + +Checkpoints carry format version, session, sequence, exact length, commit trailer, and CRC-32C. +Restore validates into temporary state before replacing live books. Gap fill has request/response +CRCs, exact range coverage, session fencing, timeouts, range limits, bounded concurrency, and a +sparse index that refreshes as the WAL grows. +`Flush(true)` is the portable OS boundary; controller caches still require power-loss protection. + ## Core data structures The order-level matching book combines three structures: @@ -95,6 +115,7 @@ are generated from the committed JSON; methodology and raw-artifact boundaries a | Batched SPSC hand-off | 6.3 ns/item median | 0.066 B/item including harness setup | | Matching at 100,000 resting orders | 98.7 ns/cycle median | state preserving | | Committed NASDAQ samples | 39,998 transitions per implementation | exact | +| Seal + journal feed packet | 813.7 ns median | 0 B/op; OS-buffered acknowledgement | | Loopback multicast, 500 subscribers | 485,393 delivered msg/s | 0 gaps; 0 CRC failures | @@ -107,28 +128,7 @@ contemporaneous relationship with returns in the recorded AMZN session but less R². Stylized-fact checks also reject the simulator as a realistic price model. It is a deterministic systems load generator; real market data remains the oracle for distribution-dependent research. -### Scaling with the audience (pre-v2 generation) - -The table above measures the v2 protocol. How the architecture scales with the *number of -subscribers* is a separate question, measured before v2 on a different host — so these figures are -not comparable with the ones above and are never combined with them. - - -| Transport | Highest sustained subscribers | Messages/s | Server CPU | Server CPU per message | -|---|---|---|---|---| -| Unicast gRPC | 900 | 86,621 | 229.4% | **26.48 µs** | -| Multicast | 6,000 | 594,067 | 73.2% | **1.23 µs** | - -Multicast delivers each message for **21× less server CPU**, to **6.7× the subscribers** at **6.9× the throughput**. - - -TCP fan-out costs one write per subscriber per update, so unicast latency tracks audience size -rather than workload: at a fixed 10,000 msg/s, 100 subscribers see 1.61 ms and 1,000 see 18.63 ms. -Multicast removes the term outright — the publisher's packet rate stays between 98.6 and 100.3/s -from 100 subscribers to 6,000, because it sends once and holds no subscriber table at all. Multicast -sustained 6,000 subscribers at 34.4 ms mean with zero gaps and zero stale receivers; 8,000 failed -with 594 detected sequence gaps, which is the sequencing machinery doing its job rather than -silently corrupting a book. Full sweeps, repeatability and threats to validity are in +Earlier audience-scaling measurements are retained under a separate host and protocol boundary in [BENCHMARKS.md](BENCHMARKS.md#transport-scaling-pre-v2-generation). ## Build and run @@ -151,6 +151,7 @@ Run the evidence suites: ```bash dotnet run --project Bench -c Release -- protocol --iterations 1000000 --trials 7 +dotnet run --project Bench -c Release -- durability --records 5000 --trials 5 dotnet run --project Bench -c Release -- queue --items 1000000 --trials 7 dotnet run --project Bench -c Release -- matching --sizes 100,1000,10000,100000 dotnet run --project Bench -c Release -- replay --data data/sample --trials 5 @@ -159,7 +160,8 @@ python3 bench/run_multicast.py --subscribers 100 500 --rates 500 --tag protocolv `ServerConfiguration` rejects invalid ports, duplicate instruments, unsafe depth and price bands, non-finite rates, multicast addresses, aliased A/B endpoints, and invalid recovery intervals. A -fixed `Seed` makes instrument flow reproducible. +fixed `Seed` makes instrument flow reproducible. `Multicast.Journal` enables the WAL and optional +retransmission port; every server start writes a distinct session directory. ## Real data @@ -182,6 +184,9 @@ and redistribution terms explicitly. Common/Matching sequencer-facing order book and depth projection Common/Books aggregated depth structures Common/Feed wire protocol, CRC, multicast, decoder state machine +Common/Durability journal, checkpoints, sparse range reads, retransmission +Common/Governance schema fingerprints, compatibility, negotiation +Common/Reference effective-dated instruments and venue sessions Common/Lobster exact integer parser and replay oracle Common/Analytics streaming microstructure statistics Server deterministic simulator and validated configuration @@ -193,9 +198,11 @@ Tests deterministic, adversarial, differential, and real-data tests A venue or trading platform would still require: -- a durable sequencer, write-ahead journal, checkpoints, catch-up, and retransmission; -- schema governance, compatibility tests, reference-data lifecycle, and session calendars; -- pre-trade risk, credit limits, kill switches, drop copy, audit retention, and entitlements; +- a replicated and fenced sequencer, quorum commit, deterministic takeover, and cross-site archive; +- retention policy, sequencer-thread checkpoints, directory-metadata durability, and repair tooling; +- runtime schema negotiation, controlled rollout, and authoritative reference-data distribution; +- pre-trade risk, credit limits, kill switches, drop copy, audit retention, and authenticated + entitlements for live and recovery channels; - PTP-synchronized clocks with uncertainty, hardware timestamps, CPU/NUMA affinity, and NIC/kernel bypass where measurements justify them; - redundant hosts and sites, deterministic failover, SLOs, telemetry, chaos drills, and disaster @@ -205,9 +212,13 @@ Those are explicit next boundaries, not implications of a low local benchmark nu ## Design lineage -The design applies ideas discussed in Jane Street's *Signals and Threads*: sequenced broadcast and -redundant lines from [Multicast and the Markets](https://signalsandthreads.com/multicast-and-the-markets/), -deterministic replay from [State Machine Replication](https://signalsandthreads.com/state-machine-replication-and-why-you-should-care/), -hostile deterministic tests from [Why Testing Is Hard](https://signalsandthreads.com/why-testing-is-hard-and-how-to-fix-it/), -measurement discipline from [Performance Engineering on Hard Mode](https://signalsandthreads.com/performance-engineering-on-hard-mode/), -and monotonic-time semantics from [Clock Synchronization](https://signalsandthreads.com/clock-synchronization/). +| Source | Applied constraint | +|---|---| +| [Signals and Threads: multicast](https://signalsandthreads.com/multicast-and-the-markets/) and [state-machine replication](https://signalsandthreads.com/state-machine-replication-and-why-you-should-care/) | global sequence, A/B lines, deterministic replay | +| [Jane Street: battle-tested systems](https://blog.janestreet.com/getting-from-tested-to-battle-tested/) | deterministic faults, simulated timing, state-machine invariants | +| [Cloudflare: million-packet UDP](https://blog.cloudflare.com/how-to-receive-a-million-packets/) | bounded socket work and a separate reliable repair channel | +| [Stripe: idempotency](https://stripe.com/blog/idempotency) and [rate limiters](https://stripe.com/blog/rate-limiters) | session identity, exact retries, cheap refusal, concurrency limits | +| [Netflix: performance under load](https://netflixtechblog.com/performance-under-load-3e6fa9a60581) | bound in-flight recovery before latency collapses | +| [Dan Luu: fsync failures](https://danluu.com/fsyncgate/) and [Mechanical Sympathy: false sharing](https://mechanical-sympathy.blogspot.com/2011/07/) | failure-specific durability claims and padded hand-off cursors | +| [HRT: devirtualisation](https://www.hudsonrivertrading.com/hrtbeat/optimising-compiler-performance-a-case-for-devirtualisation/) and [thenumb.at: open addressing](https://thenumb.at/Hashtables/) | compiler-visible hot paths and cache-coherent indexing | +| [Aeron Archive](https://aeron.io/docs/aeron-archive/overview/) and [Two Sigma metrics](https://www.twosigma.com/articles/building-a-high-throughput-metrics-system-using-open-source-software/) | position-based replay and measurement-led requirements | diff --git a/Server/Configuration/ServerConfiguration.cs b/Server/Configuration/ServerConfiguration.cs index 61b96f9..96f67df 100644 --- a/Server/Configuration/ServerConfiguration.cs +++ b/Server/Configuration/ServerConfiguration.cs @@ -2,6 +2,7 @@ using System.Text.Json; using MarketData.Common; using MarketData.Common.Feed; +using MarketData.Common.Durability; namespace MarketData.Server.Configuration { @@ -16,6 +17,17 @@ public sealed class MulticastConfiguration public int MaxBatch { get; set; } = 1; public double FlushIntervalMs { get; set; } public double SnapshotIntervalSeconds { get; set; } = 1.0; + public JournalConfiguration Journal { get; set; } = new JournalConfiguration(); + } + + public sealed class JournalConfiguration + { + public bool Enabled { get; set; } + public string Directory { get; set; } = "journal"; + public string Policy { get; set; } = nameof(DurabilityPolicy.SyncPeriodic); + public long SegmentBytes { get; set; } = 64L * 1024 * 1024; + public double SyncIntervalMs { get; set; } = 200; + public int RetransmissionPort { get; set; } } public sealed class ServerConfiguration @@ -112,6 +124,9 @@ public void Validate() if (Multicast.SnapshotIntervalSeconds > TimeSpan.MaxValue.TotalSeconds) throw new InvalidDataException("Multicast.SnapshotIntervalSeconds exceeds TimeSpan capacity"); + Multicast.Journal ??= new JournalConfiguration(); + ValidateJournal(Multicast.Journal); + if (string.IsNullOrWhiteSpace(Multicast.RedundantGroup)) { if (Multicast.RedundantPort != 0) @@ -130,6 +145,41 @@ public void Validate() throw new InvalidDataException("multicast A and B endpoints must differ"); } + private static void ValidateJournal(JournalConfiguration journal) + { + if (!journal.Enabled) + { + if (journal.RetransmissionPort != 0) + throw new InvalidDataException( + "Multicast.Journal.RetransmissionPort requires the journal"); + return; + } + + if (string.IsNullOrWhiteSpace(journal.Directory)) + throw new InvalidDataException("Multicast.Journal.Directory is required"); + try { _ = Path.GetFullPath(journal.Directory); } + catch (Exception error) when (error is ArgumentException or NotSupportedException) + { + throw new InvalidDataException("Multicast.Journal.Directory is invalid", error); + } + + if (string.IsNullOrWhiteSpace(journal.Policy) || + int.TryParse(journal.Policy, out _) || + !Enum.TryParse(journal.Policy, true, out var policy) || + !Enum.IsDefined(policy)) + throw new InvalidDataException("Multicast.Journal.Policy is invalid"); + + var minimumSegment = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + if (journal.SegmentBytes < minimumSegment) + throw new InvalidDataException("Multicast.Journal.SegmentBytes is too small"); + if (!double.IsFinite(journal.SyncIntervalMs) || journal.SyncIntervalMs <= 0 || + journal.SyncIntervalMs > uint.MaxValue - 1) + throw new InvalidDataException("Multicast.Journal.SyncIntervalMs is invalid"); + if (journal.RetransmissionPort != 0) + ValidatePort(journal.RetransmissionPort, "Multicast.Journal.RetransmissionPort"); + } + private static bool IsNonNegativeFinite(double value) => double.IsFinite(value) && value >= 0; private const double MaxTimerMilliseconds = 4_294_967_294; diff --git a/Server/Server.cs b/Server/Server.cs index 8ba0f69..01020a0 100644 --- a/Server/Server.cs +++ b/Server/Server.cs @@ -9,6 +9,8 @@ using System.Linq; using System.Text; using System.Threading.Tasks; +using MarketData.Common.Durability; +using MarketData.Common.Feed; namespace MarketData.Server { @@ -24,30 +26,52 @@ public Server(ServerConfiguration config) new Orderbook(instrument, _service.RegisterProducer(), config.PriceBand, config.Seed)); } - /// - /// Selects the dissemination transport. The two are interchangeable behind - /// , which is what makes them directly comparable: the - /// matching engine above is byte-for-byte the same in both configurations, so a - /// measured difference is attributable to the transport and nothing else. - /// + /// Selects gRPC fan-out or multicast without changing the matching path. private IOrderbookService CreateService(ServerConfiguration config) { if (!config.Multicast.Enabled) return new OrderbookService(config.Port, this, config.VerboseLogging, config.SubscriberQueueCapacity, config.UseRingQueue); - return new MulticastOrderbookService( - IPAddress.Parse(config.Multicast.Group), - config.Multicast.Port, - IPAddress.Parse(config.Multicast.Interface), - config.Multicast.MaxBatch, - TimeSpan.FromMilliseconds(config.Multicast.FlushIntervalMs), - TimeSpan.FromSeconds(config.Multicast.SnapshotIntervalSeconds), - this, - string.IsNullOrWhiteSpace(config.Multicast.RedundantGroup) - ? null - : IPAddress.Parse(config.Multicast.RedundantGroup), - config.Multicast.RedundantPort); + var journal = CreateJournal(config.Multicast.Journal); + + try + { + return new MulticastOrderbookService( + IPAddress.Parse(config.Multicast.Group), + config.Multicast.Port, + IPAddress.Parse(config.Multicast.Interface), + config.Multicast.MaxBatch, + TimeSpan.FromMilliseconds(config.Multicast.FlushIntervalMs), + TimeSpan.FromSeconds(config.Multicast.SnapshotIntervalSeconds), + this, + string.IsNullOrWhiteSpace(config.Multicast.RedundantGroup) + ? null + : IPAddress.Parse(config.Multicast.RedundantGroup), + config.Multicast.RedundantPort, + journal, + config.Multicast.Journal.RetransmissionPort); + } + catch + { + journal?.Dispose(); + throw; + } + } + + private static WriteAheadJournal CreateJournal(JournalConfiguration configuration) + { + if (!configuration.Enabled) + return null; + + var sessionId = MulticastPublisher.NewSessionId(); + var directory = Path.Combine(Path.GetFullPath(configuration.Directory), + $"session-{sessionId:X16}"); + var policy = Enum.Parse(configuration.Policy, ignoreCase: true); + + return new WriteAheadJournal(directory, sessionId, policy, + configuration.SegmentBytes, TimeSpan.FromMilliseconds(configuration.SyncIntervalMs), + initialSequence: 0); } public IReadOnlyCollection InstrumentIds => _orderbooks.Keys; diff --git a/Tests/ConcurrencyRegressionTests.cs b/Tests/ConcurrencyRegressionTests.cs index 4c9e464..ace563e 100644 --- a/Tests/ConcurrencyRegressionTests.cs +++ b/Tests/ConcurrencyRegressionTests.cs @@ -107,17 +107,17 @@ public async Task CountNeverGoesNegativeWhileBothCursorsMove() var negatives = 0; var aboveCapacity = 0; - var producer = Task.Run(() => + var producer = Task.Factory.StartNew(() => { while (!cancellationToken.IsCancellationRequested) ring.TryWrite(1); - }, cancellationToken); + }, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default); - var consumer = Task.Run(() => + var consumer = Task.Factory.StartNew(() => { while (!cancellationToken.IsCancellationRequested) ring.TryRead(out _); - }, cancellationToken); + }, cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default); while (!cancellationToken.IsCancellationRequested) { diff --git a/Tests/ConfigurationTests.cs b/Tests/ConfigurationTests.cs index 8d9ba0b..42af50f 100644 --- a/Tests/ConfigurationTests.cs +++ b/Tests/ConfigurationTests.cs @@ -1,5 +1,6 @@ using MarketData.Common; using MarketData.Common.Feed; +using MarketData.Common.Durability; using MarketData.Server.Configuration; using Xunit; @@ -82,6 +83,31 @@ public void RedundantMulticastEndpointMustBePresentAndDistinct() config.Validate(); } + [Fact] + public void JournalAndRetransmissionConfigurationIsValidated() + { + var config = Valid(); + config.Multicast.Enabled = true; + config.Multicast.Journal.RetransmissionPort = 32001; + Assert.Throws(config.Validate); + + config.Multicast.Journal.Enabled = true; + config.Multicast.Journal.Policy = "not-a-policy"; + Assert.Throws(config.Validate); + + config.Multicast.Journal.Policy = "1"; + Assert.Throws(config.Validate); + + config.Multicast.Journal.Policy = "SyncPeriodic"; + config.Multicast.Journal.SyncIntervalMs = double.NaN; + Assert.Throws(config.Validate); + + config.Multicast.Journal.SyncIntervalMs = 10; + config.Multicast.Journal.SegmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + config.Validate(); + } + private static ServerConfiguration Valid() => new ServerConfiguration { Instruments = new[] { Instrument(1, "TEST") }, diff --git a/Tests/DurabilityTests.cs b/Tests/DurabilityTests.cs index 71628bd..ac3591c 100644 --- a/Tests/DurabilityTests.cs +++ b/Tests/DurabilityTests.cs @@ -1,18 +1,20 @@ using System; using System.Collections.Generic; +using System.Diagnostics; using System.IO; using System.Linq; +using System.Net; using System.Text; +using System.Threading; using System.Threading.Tasks; using MarketData.Common.Books; using MarketData.Common.Durability; +using MarketData.Common.Feed; using Xunit; namespace MarketData.Tests { - /// - /// The durability layer, tested the only way that means anything: by damaging it. - /// + /// Durability, corruption, restart, and gap-fill invariants. public sealed class DurabilityTests : IDisposable { private readonly string _root = Path.Combine(Path.GetTempPath(), @@ -36,6 +38,25 @@ private string Dir(string name) private static byte[] Payload(int n) => Encoding.UTF8.GetBytes($"message-{n:D6}"); + private static byte[] SnapshotPacket(ulong sequence, ulong session = Session) + { + var packet = new byte[FeedProtocol.HeaderSize + FeedProtocol.SnapshotSize(0, 0)]; + var size = FeedProtocol.HeaderSize + FeedProtocol.WriteSnapshot( + packet.AsSpan(FeedProtocol.HeaderSize), 1, ReadOnlySpan.Empty, + ReadOnlySpan.Empty); + FeedProtocol.WriteHeader(packet.AsSpan(0, size), 1, session, sequence, 1); + return packet.AsSpan(0, size).ToArray(); + } + + private static byte[] IncrementalPacket(ulong sequence, int price, ulong session = Session) + { + var packet = new byte[FeedProtocol.HeaderSize + FeedProtocol.IncrementalSize]; + FeedProtocol.WriteIncremental(packet.AsSpan(FeedProtocol.HeaderSize), FeedMessageType.Add, + 1, Side.Bid, new PriceLevel(price, 100)); + FeedProtocol.WriteHeader(packet, 1, session, sequence, 1); + return packet; + } + // ------------------------------------------------------------------ framing [Fact] @@ -54,11 +75,7 @@ public void ARecordRoundTrips() Assert.Equal(buffer.Length, record.TotalSize); } - /// Every single-byte truncation must be reported as incomplete, never as valid. - /// - /// This is the case a crash actually produces, and the one a naive reader gets wrong: it - /// finds a plausible header, trusts the length, and reads past the end of what was written. - /// + /// Every truncated prefix is incomplete. [Fact] public void EveryTruncationIsDetected() { @@ -251,7 +268,8 @@ public void SegmentsRotateAndAllOfThemAreRead() var directory = Dir("segments"); var payload = Payload(1); var perSegment = 20; - var segmentBytes = JournalRecord.OverheadSize + JournalRecord.MaxPayloadSize; + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); using (var journal = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered, segmentBytes)) @@ -290,14 +308,7 @@ public void EveryDurabilityPolicyRecoversWhatItReturnedFrom(DurabilityPolicy pol // ------------------------------------------------------------------ checkpoints - /// - /// Checkpoint + subsequent journal must equal full replay, exactly. - /// - /// - /// The invariant the whole recovery story rests on. Asserted against a book rebuilt from - /// scratch rather than against itself, because a checkpoint that is confidently wrong is - /// worse than no checkpoint at all. - /// + /// Checkpoint plus tail replay equals full replay. [Fact] public void ACheckpointPlusTheRestOfTheLogEqualsAFullReplay() { @@ -323,6 +334,8 @@ public void ACheckpointPlusTheRestOfTheLogEqualsAFullReplay() using (var journal = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered)) { + var encoded = new byte[13]; + for (var i = 0; i < operations.Count; i++) { var op = operations[i]; @@ -333,11 +346,10 @@ public void ACheckpointPlusTheRestOfTheLogEqualsAFullReplay() book.Upsert(op.Side, op.Price, op.Quantity); - Span encoded = stackalloc byte[13]; System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(encoded, op.Instrument); encoded[4] = (byte)op.Side; - System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(encoded.Slice(5), op.Price); - System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(encoded.Slice(9), op.Quantity); + System.Buffers.Binary.BinaryPrimitives.WriteInt32LittleEndian(encoded.AsSpan(5), op.Price); + System.Buffers.Binary.BinaryPrimitives.WriteUInt32LittleEndian(encoded.AsSpan(9), op.Quantity); journal.Append(JournalRecordType.Message, sequence, i, encoded); @@ -426,7 +438,10 @@ public void PruningKeepsTheNewestCheckpointsAndNeverAllOfThem() books[1].Upsert(Side.Bid, 10, 100); for (ulong sequence = 1; sequence <= 6; sequence++) + { + journal.Append(JournalRecordType.Message, sequence, 0, ReadOnlySpan.Empty); Checkpoint.Write(checkpoints, journal, sequence, Session, books); + } } Assert.Equal(3, Checkpoint.Prune(checkpoints, keep: 3)); @@ -438,22 +453,14 @@ public void PruningKeepsTheNewestCheckpointsAndNeverAllOfThem() Assert.Throws(() => Checkpoint.Prune(checkpoints, keep: 0)); } - /// - /// Recovering from a checkpoint must not re-read the history behind it. - /// - /// - /// The point of a checkpoint is that recovery stops being proportional to uptime. An - /// implementation that restores the checkpoint and *then* scans the whole log anyway is - /// still O(uptime) and has bought nothing but the book replay - which is what the first - /// version here did, and what the recovery benchmark exposed. This asserts the skip - /// happens, by counting the records the scan actually visits. - /// + /// Checkpoint recovery skips complete historical segments. [Fact] public void RecoveringFromACheckpointSkipsSegmentsBelowIt() { var directory = Dir("skip-journal"); var payload = new byte[512]; - var segmentBytes = JournalRecord.OverheadSize + JournalRecord.MaxPayloadSize; + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); const int count = 4_000; using (var journal = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered, @@ -499,7 +506,8 @@ public void SegmentSkippingNeverLosesARecordInRange() { var directory = Dir("skip-correct"); var payload = new byte[512]; - var segmentBytes = JournalRecord.OverheadSize + JournalRecord.MaxPayloadSize; + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); const int count = 3_000; using (var journal = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered, @@ -550,7 +558,8 @@ public async Task AGapIsFilledFromTheJournal() service.Start(); var client = new RetransmissionClient(service.Port); - var recovered = await client.RequestAsync(100, 109); + var recovered = await client.RequestAsync(100, 109, + TestContext.Current.CancellationToken); Assert.NotNull(recovered); Assert.Equal(10, recovered.Count); @@ -560,14 +569,7 @@ public async Task AGapIsFilledFromTheJournal() Assert.Equal(Payload(100 + i), recovered[i].Payload); } - /// - /// A gap too large to fill from history is refused, not served slowly. - /// - /// - /// Retransmission is where one struggling subscriber can become everybody's problem. A - /// subscriber this far behind should take a snapshot, which costs O(book) rather than - /// O(history). - /// + /// Oversized recovery ranges require a snapshot. [Fact] public async Task AnOversizedRequestIsRefusedRatherThanServed() { @@ -580,7 +582,8 @@ public async Task AnOversizedRequestIsRefusedRatherThanServed() service.Start(); var client = new RetransmissionClient(service.Port); - var refused = await client.RequestAsync(1, RetransmissionService.MaxRangeLength + 1); + var refused = await client.RequestAsync(1, RetransmissionService.MaxRangeLength + 1, + TestContext.Current.CancellationToken); Assert.Null(refused); Assert.Equal(1, service.RequestsRefused); @@ -604,10 +607,11 @@ public async Task GapFillWorksWhileThePublisherIsStillWriting() { for (var i = 51; i <= 400; i++) journal.Append(JournalRecordType.Message, (ulong)i, i, Payload(i)); - }); + }, TestContext.Current.CancellationToken); var client = new RetransmissionClient(service.Port); - var recovered = await client.RequestAsync(10, 19); + var recovered = await client.RequestAsync(10, 19, + TestContext.Current.CancellationToken); await publishing; @@ -615,5 +619,500 @@ public async Task GapFillWorksWhileThePublisherIsStillWriting() Assert.Equal(10, recovered.Count); Assert.Equal(Payload(10), recovered[0].Payload); } + + [Fact] + public void ReopeningResumesTheRecoveredWatermark() + { + var directory = Dir("resume"); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + for (ulong sequence = 1; sequence <= 10; sequence++) + journal.Append(JournalRecordType.Message, sequence, 0, Payload((int)sequence)); + } + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + Assert.Equal(11UL, journal.NextSequence); + Assert.Equal(10UL, journal.LastSequence); + Assert.Throws(() => + journal.Append(JournalRecordType.Message, 10, 0, Payload(10))); + journal.Append(JournalRecordType.Message, 11, 0, Payload(11)); + } + + var report = JournalReader.Recover(directory); + Assert.Equal(RecoveryOutcome.Clean, report.Outcome); + Assert.Equal(11UL, report.LastSequence); + Assert.Equal(12UL, report.NextSequence); + } + + [Fact] + public void ReopeningRefusesADifferentInitialSequence() + { + var directory = Dir("resume-initial"); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 1)) + journal.Append(JournalRecordType.Message, 1, 0, Payload(1)); + + Assert.Throws(() => new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0)); + } + + [Fact] + public void ReopeningReplacesAnIncompleteFirstSegment() + { + var directory = Dir("repair-first-header"); + + using (new WriteAheadJournal(directory, Session, DurabilityPolicy.SyncEachRecord)) { } + + var segment = Directory.GetFiles(directory, "segment-*.jrn").Single(); + using (var stream = new FileStream(segment, FileMode.Open, FileAccess.Write)) + stream.SetLength(4); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + Assert.Equal(1UL, journal.NextSequence); + journal.AppendNext(0, Payload(1)); + } + + var report = JournalReader.Recover(directory); + Assert.Equal(RecoveryOutcome.Clean, report.Outcome); + Assert.Equal(1UL, report.LastSequence); + } + + [Fact] + public void ReopeningRepairsATornTailBeforeAppending() + { + var directory = Dir("repair-tail"); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + for (ulong sequence = 1; sequence <= 10; sequence++) + journal.Append(JournalRecordType.Message, sequence, 0, Payload((int)sequence)); + } + + var segment = Directory.GetFiles(directory, "segment-*.jrn").Single(); + using (var stream = new FileStream(segment, FileMode.Open, FileAccess.Write)) + stream.SetLength(stream.Length - JournalRecord.SizeFor(Payload(10).Length) / 2); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + Assert.Equal(10UL, journal.NextSequence); + journal.Append(JournalRecordType.Message, 10, 0, Payload(10)); + } + + var report = JournalReader.Recover(directory); + Assert.Equal(RecoveryOutcome.Clean, report.Outcome); + Assert.Equal(10UL, report.LastSequence); + } + + [Fact] + public void ReopeningRefusesCommittedCorruption() + { + var directory = Dir("refuse-corrupt"); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + { + journal.Append(JournalRecordType.Message, 1, 0, Payload(1)); + } + + var segment = Directory.GetFiles(directory, "segment-*.jrn").Single(); + var bytes = File.ReadAllBytes(segment); + bytes[bytes.Length - JournalRecord.TrailerSize - 1] ^= 1; + File.WriteAllBytes(segment, bytes); + + Assert.Throws(() => + new WriteAheadJournal(directory, Session, DurabilityPolicy.SyncEachRecord)); + } + + [Fact] + public void ADeletedMiddleSegmentIsCorruption() + { + var directory = Dir("missing-segment"); + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + var payload = new byte[JournalRecord.MaxPayloadSize]; + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered, segmentBytes)) + { + for (ulong sequence = 1; sequence <= 4; sequence++) + journal.Append(JournalRecordType.Message, sequence, 0, payload); + } + + var segments = Directory.GetFiles(directory, "segment-*.jrn").OrderBy(x => x).ToArray(); + Assert.True(segments.Length >= 4); + File.Delete(segments[1]); + + var report = JournalReader.Recover(directory); + Assert.Equal(RecoveryOutcome.Corrupt, report.Outcome); + Assert.Equal(JournalReadResult.SegmentOrder, report.Failure); + } + + [Fact] + public void ADeletedFirstSegmentIsCorruption() + { + var directory = Dir("missing-first-segment"); + var segmentBytes = JournalRecord.SizeFor(16) + + JournalRecord.SizeFor(JournalRecord.MaxPayloadSize); + var payload = new byte[JournalRecord.MaxPayloadSize]; + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered, segmentBytes)) + { + journal.AppendNext(0, payload); + journal.AppendNext(0, payload); + } + + var segments = Directory.GetFiles(directory, "segment-*.jrn").OrderBy(x => x).ToArray(); + Assert.True(segments.Length >= 2); + File.Delete(segments[0]); + + var report = JournalReader.Recover(directory); + Assert.Equal(RecoveryOutcome.Corrupt, report.Outcome); + Assert.Equal(JournalReadResult.SegmentOrder, report.Failure); + } + + [Fact] + public void ASecondWriterCannotAcquireTheJournal() + { + var directory = Dir("writer-lease"); + using var first = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered); + + Assert.Throws(() => + new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered)); + } + + [Fact] + public async Task PeriodicDurabilitySyncsAnIdleWriter() + { + var directory = Dir("periodic-idle"); + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncPeriodic, syncInterval: TimeSpan.FromMilliseconds(10)); + journal.Sync(); + var before = journal.Syncs; + journal.Append(JournalRecordType.Message, 1, 0, Payload(1)); + + var deadline = Stopwatch.GetTimestamp() + Stopwatch.Frequency; + while (journal.Syncs == before && Stopwatch.GetTimestamp() < deadline) + await Task.Delay(10, TestContext.Current.CancellationToken); + + Assert.True(journal.Syncs > before, "the idle periodic writer never reached storage"); + } + + [Fact] + public void OsBufferedAppendAllocatesNothingInSteadyState() + { + var directory = Dir("append-allocation"); + var payload = new byte[64]; + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered); + + journal.AppendNext(0, payload); + journal.AppendNext(0, payload); + var before = GC.GetAllocatedBytesForCurrentThread(); + + for (var i = 0; i < 1_000; i++) + journal.AppendNext(i, payload); + + var allocated = GC.GetAllocatedBytesForCurrentThread() - before; + Assert.Equal(0, allocated); + } + + [Fact] + public void FeedPacketsAdvanceByTheirMessageCount() + { + var directory = Dir("packet-range"); + var packet = new byte[FeedProtocol.HeaderSize + 2 * FeedProtocol.IncrementalSize]; + var offset = FeedProtocol.HeaderSize; + offset += FeedProtocol.WriteIncremental(packet.AsSpan(offset), FeedMessageType.Add, 1, + Side.Bid, new PriceLevel(-1, 100)); + offset += FeedProtocol.WriteIncremental(packet.AsSpan(offset), FeedMessageType.Add, 1, + Side.Bid, new PriceLevel(-2, 100)); + FeedProtocol.WriteHeader(packet.AsSpan(0, offset), 2, Session, 0, 1); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0)) + { + journal.AppendPacket(packet.AsSpan(0, offset)); + Assert.Equal(2UL, journal.NextSequence); + } + + var result = JournalReader.TryReadRange(directory, Session, 0, 1, out var found); + Assert.Equal(JournalRangeResult.Success, result); + Assert.Single(found); + Assert.Equal(2, found[0].MessageCount); + Assert.Equal(packet.AsSpan(0, offset).ToArray(), found[0].Payload); + } + + [Fact] + public void GapFillDoesNotReturnAPartialFeedPacket() + { + var directory = Dir("packet-alignment"); + var packet = new byte[FeedProtocol.HeaderSize + 2 * FeedProtocol.IncrementalSize]; + var offset = FeedProtocol.HeaderSize; + offset += FeedProtocol.WriteIncremental(packet.AsSpan(offset), FeedMessageType.Add, 1, + Side.Bid, new PriceLevel(-1, 100)); + offset += FeedProtocol.WriteIncremental(packet.AsSpan(offset), FeedMessageType.Add, 1, + Side.Bid, new PriceLevel(-2, 100)); + FeedProtocol.WriteHeader(packet.AsSpan(0, offset), 2, Session, 0, 1); + + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0)) + journal.AppendPacket(packet.AsSpan(0, offset)); + + var result = JournalReader.TryReadRange(directory, Session, 1, 1, out var found); + + Assert.Equal(JournalRangeResult.Missing, result); + Assert.Empty(found); + } + + [Fact] + public void PublisherRejectsAnInvalidBatchBound() + { + Assert.Throws(() => new MulticastPublisher( + IPAddress.Parse("239.7.7.77"), 31777, IPAddress.Loopback, maxBatch: 0)); + } + + [Fact] + public void PublisherPersistsTheSealedPacket() + { + var directory = Dir("publisher-journal"); + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0); + + using (var publisher = new MulticastPublisher(IPAddress.Parse("239.7.7.77"), 31777, + IPAddress.Loopback, maxBatch: 1, sessionId: Session, journal: journal)) + { + publisher.PublishSnapshot(1, ReadOnlySpan.Empty, + ReadOnlySpan.Empty); + publisher.Flush(); + } + + var records = new List(); + var report = JournalReader.Recover(directory, (in JournalRecordView record) => + { + if (record.Type == JournalRecordType.FeedPacket) + records.Add(record.Payload.ToArray()); + return true; + }); + + Assert.Equal(RecoveryOutcome.Clean, report.Outcome); + Assert.Single(records); + Assert.True(FeedProtocol.TryReadHeader(records[0], out var header, out _)); + Assert.Equal(Session, header.SessionId); + Assert.Equal(0UL, header.FirstSequence); + } + + [Fact] + public async Task FeedGapRepairReplaysPacketsBeforeTheHeldLivePacket() + { + var directory = Dir("feed-repair"); + var snapshot = SnapshotPacket(0); + var first = IncrementalPacket(1, -1); + var second = IncrementalPacket(2, -2); + + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0); + journal.AppendPacket(snapshot); + journal.AppendPacket(first); + journal.AppendPacket(second); + + using var service = new RetransmissionService(directory); + service.Start(); + var decoder = new FeedDecoder(_ => new SortedArrayBook(10)); + decoder.Consume(snapshot); + var recovery = new FeedRecoveryCoordinator(decoder, + new RetransmissionClient(service.Port)); + + var result = await recovery.ConsumeAsync(second, TestContext.Current.CancellationToken); + + Assert.Equal(GapRecoveryResult.Repaired, result); + Assert.False(decoder.IsStale); + Assert.Equal(3UL, decoder.ExpectedSequence); + Assert.Equal(new[] { -1, -2 }, + decoder.BookFor(1).ToList(Side.Bid).Select(level => level.Price)); + Assert.Equal(0, decoder.Statistics.Gaps); + } + + [Fact] + public async Task MissingGapFillRequiresASnapshot() + { + var directory = Dir("feed-repair-missing"); + var snapshot = SnapshotPacket(0); + var future = IncrementalPacket(2, -2); + + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0); + journal.AppendPacket(snapshot); + + using var service = new RetransmissionService(directory); + service.Start(); + var decoder = new FeedDecoder(_ => new SortedArrayBook(10)); + decoder.Consume(snapshot); + var recovery = new FeedRecoveryCoordinator(decoder, + new RetransmissionClient(service.Port)); + + var result = await recovery.ConsumeAsync(future, TestContext.Current.CancellationToken); + + Assert.Equal(GapRecoveryResult.SnapshotRequired, result); + Assert.True(decoder.IsStale); + Assert.Equal(1, decoder.Statistics.Gaps); + } + + [Fact] + public async Task RetransmissionRefusesOverflowAndWrongSession() + { + var directory = Dir("request-validation"); + using (var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.SyncEachRecord)) + journal.Append(JournalRecordType.Message, 1, 0, Payload(1)); + + using var service = new RetransmissionService(directory); + service.Start(); + var client = new RetransmissionClient(service.Port); + + var overflow = await client.RequestDetailedAsync(Session, 0, ulong.MaxValue, + TestContext.Current.CancellationToken); + var wrongSession = await client.RequestDetailedAsync(Session + 1, 1, 1, + TestContext.Current.CancellationToken); + + Assert.Equal(RetransmissionStatus.InvalidRequest, overflow.Status); + Assert.Equal(RetransmissionStatus.WrongSession, wrongSession.Status); + Assert.Equal(2, service.RequestsRefused); + } + + [Fact] + public async Task RetransmissionReportsLiveTailCorruption() + { + var directory = Dir("request-corrupt-tail"); + using var journal = new WriteAheadJournal(directory, Session, + DurabilityPolicy.OsBuffered); + journal.AppendNext(0, Payload(1)); + + using var service = new RetransmissionService(directory); + service.Start(); + journal.AppendNext(0, Payload(2)); + + var segment = Directory.GetFiles(directory, "segment-*.jrn").Single(); + using (var stream = new FileStream(segment, FileMode.Open, FileAccess.ReadWrite, + FileShare.ReadWrite)) + { + stream.Position = stream.Length - JournalRecord.TrailerSize - 1; + var value = stream.ReadByte(); + stream.Position--; + stream.WriteByte((byte)(value ^ 1)); + stream.Flush(); + } + + var response = await new RetransmissionClient(service.Port).RequestDetailedAsync( + Session, 1, 2, TestContext.Current.CancellationToken); + + Assert.Equal(RetransmissionStatus.CorruptJournal, response.Status); + Assert.Empty(response.Messages); + } + + [Fact] + public void CheckpointsDetectEverySingleBitCorruption() + { + var journalDirectory = Dir("checkpoint-crc-journal"); + var checkpointDirectory = Dir("checkpoint-crc"); + string path; + + using (var journal = new WriteAheadJournal(journalDirectory, Session, + DurabilityPolicy.SyncEachRecord)) + { + journal.Append(JournalRecordType.Message, 1, 0, Payload(1)); + var book = new SortedArrayBook(4); + book.Upsert(Side.Bid, -1, 100); + book.Upsert(Side.Ask, 1, 100); + path = Checkpoint.Write(checkpointDirectory, journal, 1, Session, + new Dictionary { [1] = book }); + } + + var original = File.ReadAllBytes(path); + + for (var index = 0; index < original.Length; index++) + { + for (var bit = 0; bit < 8; bit++) + { + var damaged = (byte[])original.Clone(); + damaged[index] ^= (byte)(1 << bit); + File.WriteAllBytes(path, damaged); + + var target = new Dictionary + { + [99] = new SortedArrayBook(1), + }; + + Assert.Throws(() => + Checkpoint.Restore(path, _ => new SortedArrayBook(4), target, Session)); + Assert.True(target.ContainsKey(99), "failed restore mutated the target state"); + } + } + + File.WriteAllBytes(path, original); + Assert.Throws(() => Checkpoint.Restore(path, + _ => new SortedArrayBook(4), new Dictionary(), Session + 1)); + } + + [Fact] + public void ASequenceZeroCheckpointCanBeDiscovered() + { + var journalDirectory = Dir("checkpoint-zero-journal"); + var checkpointDirectory = Dir("checkpoint-zero"); + var packet = SnapshotPacket(0); + string written; + + using (var journal = new WriteAheadJournal(journalDirectory, Session, + DurabilityPolicy.SyncEachRecord, initialSequence: 0)) + { + journal.AppendPacket(packet); + written = Checkpoint.Write(checkpointDirectory, journal, 0, Session, + new Dictionary { [1] = new SortedArrayBook(4) }); + } + + Assert.Equal(written, Checkpoint.FindLatest(checkpointDirectory)); + } + + [Fact] + public void SequencerExhaustionNeverWraps() + { + var sequencer = new Sequencer(ulong.MaxValue - 1); + Assert.Equal(ulong.MaxValue, sequencer.Next()); + Assert.Throws(() => sequencer.Next()); + Assert.Equal(ulong.MaxValue, sequencer.Last); + } + + [Fact] + public void SparseRangeIndexRefreshesTheLiveTail() + { + var directory = Dir("sparse-index"); + using var journal = new WriteAheadJournal(directory, Session, DurabilityPolicy.OsBuffered); + + for (ulong sequence = 1; sequence <= 1_000; sequence++) + journal.Append(JournalRecordType.Message, sequence, 0, Payload((int)sequence)); + + var reader = new JournalRangeReader(directory, stride: 32); + Assert.True(reader.IndexEntries >= 31); + + for (ulong sequence = 1_001; sequence <= 2_000; sequence++) + journal.Append(JournalRecordType.Message, sequence, 0, Payload((int)sequence)); + + var result = reader.TryRead(Session, 1_990, 2_000, out var found); + + Assert.Equal(JournalRangeResult.Success, result); + Assert.Equal(11, found.Count); + Assert.Equal(Enumerable.Range(1990, 11).Select(value => (ulong)value), + found.Select(message => message.Sequence)); + Assert.True(reader.IndexEntries >= 62); + } } } diff --git a/Tests/GovernanceTests.cs b/Tests/GovernanceTests.cs index d61f4e8..2d1469f 100644 --- a/Tests/GovernanceTests.cs +++ b/Tests/GovernanceTests.cs @@ -54,6 +54,35 @@ public void AFingerprintChangesWhenAnyPartOfTheLayoutDoes() Assert.NotEqual(baseline, movedField.Fingerprint); } + [Fact] + public void FingerprintComponentsAreLengthDelimited() + { + var field = new[] { new SchemaField("F", FieldType.UInt8, 0, 1) }; + var left = new Schema(1, new[] { new MessageSchema("A1", 2, field) }); + var right = new Schema(1, new[] { new MessageSchema("A", 12, field) }); + + Assert.NotEqual(left.Fingerprint, right.Fingerprint); + } + + [Fact] + public void SchemaConstructionTakesAnImmutableSnapshot() + { + var fields = new List + { + new("A", FieldType.UInt8, 0, 1), + }; + var messages = new List { new("M", 1, fields) }; + var schema = new Schema(1, messages); + var fingerprint = schema.Fingerprint; + + fields.Add(new SchemaField("B", FieldType.UInt8, 1, 1)); + messages.Clear(); + + Assert.Single(schema.Messages); + Assert.Single(schema.Messages[0].Fields); + Assert.Equal(fingerprint, schema.Fingerprint); + } + /// Overlapping fields are silently destructive, so they are rejected outright. [Fact] public void OverlappingFieldsAreRejected() @@ -83,7 +112,7 @@ public void AMismatchedWidthIsRejected() // ------------------------------------------------------- compatibility rules [Fact] - public void AddingAnOptionalFieldPastTheEndIsBackwardCompatible() + public void AppendingAnOptionalFieldWithoutMessageFramingIsBreaking() { var older = new Schema(1, new[] { @@ -101,9 +130,12 @@ public void AddingAnOptionalFieldPastTheEndIsBackwardCompatible() var report = Compatibility.Compare(older, newer); - Assert.Equal(CompatibilityKind.BackwardCompatible, report.Kind); - Assert.True(report.CanDeployIndependently); - Compatibility.AssertDeployableAgainst(older, newer); + Assert.Equal(CompatibilityKind.Breaking, report.Kind); + Assert.False(report.CanDeployIndependently); + Assert.Contains(report.Breaks, + item => item.Message.Contains("without a length-delimited envelope")); + Assert.Throws( + () => Compatibility.AssertDeployableAgainst(older, newer)); } /// @@ -232,10 +264,18 @@ public static IEnumerable BreakingChanges() }; } - /// The shipped versions must actually be safe to deploy against each other. [Fact] - public void TheShippedSchemaEvolutionIsDeployable() - => Compatibility.AssertDeployableAgainst(FeedSchemas.V1, FeedSchemas.V2); + public void ANewUnframedMessageTypeRequiresAReaderFirstCutover() + { + var report = Compatibility.Compare(FeedSchemas.V1, FeedSchemas.V2); + + Assert.Equal(CompatibilityKind.ForwardCompatible, report.Kind); + Assert.False(report.CanDeployIndependently); + Assert.Contains(report.Breaks, + item => item.Message.Contains("cannot skip unframed message types")); + Assert.Throws( + () => Compatibility.AssertDeployableAgainst(FeedSchemas.V1, FeedSchemas.V2)); + } // ------------------------------------------------------- registry @@ -253,37 +293,62 @@ public void NoSharedVersionFailsRatherThanSilentlyDowngrading() => Assert.Throws( () => SchemaRegistry.Default.Negotiate(new[] { 98, 99 })); - /// - /// Two builds agreeing they speak "v2" while disagreeing about what v2 is. - /// - /// - /// The failure the fingerprint exists to catch, and the one a version number alone cannot: - /// someone edited a layout without bumping the number. - /// + /// Equal version numbers do not override a layout mismatch. [Fact] public void SameVersionDifferentLayoutIsRejected() { var thrown = Assert.Throws( - () => SchemaRegistry.Default.Confirm(2, FeedSchemas.V2.Fingerprint ^ 1)); + () => SchemaRegistry.Default.Confirm(2, FeedSchemas.V2.Fingerprint ^ (UInt128)1)); Assert.Contains("different layout", thrown.Message); Assert.Same(FeedSchemas.V2, SchemaRegistry.Default.Confirm(2, FeedSchemas.V2.Fingerprint)); } - /// - /// The declared schema must match the encoder's actual constants. - /// - /// - /// A schema that has drifted from the code it describes is worse than no schema, because it - /// will be believed. This is the check that keeps the two honest. - /// + /// The declared fixed layouts mirror the encoder. [Fact] public void TheDeclaredLayoutMatchesTheEncoder() { - var incremental = FeedSchemas.Current.Find(FeedSchemas.IncrementalTypeCode); + Assert.Equal(5, FeedSchemas.Current.Messages.Count); + AssertIncremental(FeedSchemas.AddTypeCode, "Add"); + AssertIncremental(FeedSchemas.ReplaceTypeCode, "Replace"); + AssertIncremental(FeedSchemas.RemoveTypeCode, "Remove"); + + var snapshot = FeedSchemas.Current.Find(FeedSchemas.SnapshotHeaderTypeCode); + Assert.NotNull(snapshot); + Assert.Equal(7, snapshot.Size); + Assert.Collection(snapshot.Fields.OrderBy(field => field.Offset), + field => Assert.Equal(("MessageType", FieldType.UInt8, 0, 1), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("InstrumentId", FieldType.Int32, 1, 4), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("BidLevels", FieldType.UInt8, 5, 1), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("AskLevels", FieldType.UInt8, 6, 1), + (field.Name, field.Type, field.Offset, field.Length))); + + var heartbeat = FeedSchemas.Current.Find(FeedSchemas.HeartbeatTypeCode); + Assert.NotNull(heartbeat); + Assert.Equal(1, heartbeat.Size); + } + + private static void AssertIncremental(byte typeCode, string name) + { + var incremental = FeedSchemas.Current.Find(typeCode); Assert.NotNull(incremental); + Assert.Equal(name, incremental.Name); Assert.Equal(FeedProtocol.IncrementalSize, incremental.Size); + Assert.Collection(incremental.Fields.OrderBy(field => field.Offset), + field => Assert.Equal(("MessageType", FieldType.UInt8, 0, 1), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("InstrumentId", FieldType.Int32, 1, 4), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("Price", FieldType.Int32, 5, 4), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("Quantity", FieldType.UInt32, 9, 4), + (field.Name, field.Type, field.Offset, field.Length)), + field => Assert.Equal(("Side", FieldType.UInt8, 13, 1), + (field.Name, field.Type, field.Offset, field.Length))); } } @@ -354,6 +419,50 @@ public void ABitemporalQueryReproducesWhatWasKnownAtTheTime() Assert.Equal(5, master.AsKnownAt(1, tradeDay, new DateTime(2021, 6, 1))?.TickSize); } + [Fact] + public void AnAmendmentDoesNotRewritePriorKnowledge() + { + var master = new InstrumentMaster(); + var effective = new DateTime(2021, 1, 1); + var learned = new DateTime(2021, 6, 1); + master.Amend(Record(1, "OLD", Y2020, DateTime.MaxValue, recorded: Y2020)); + master.Amend(Record(1, "NEW", effective, DateTime.MaxValue, + ReferenceChangeReason.SymbolChange, recorded: learned)); + + Assert.Equal("OLD", master.AsKnownAt(1, effective.AddMonths(1), + learned.AddTicks(-1))?.Symbol); + Assert.Equal("NEW", master.AsOf(1, effective.AddMonths(1))?.Symbol); + } + + [Fact] + public void EqualSystemTimesHaveDeterministicEffectiveOrdering() + { + var learned = new DateTime(2021, 6, 1); + var first = Record(1, "OLD", Y2020, DateTime.MaxValue, recorded: learned); + var second = Record(1, "NEW", new DateTime(2021, 1, 1), DateTime.MaxValue, + ReferenceChangeReason.SymbolChange, recorded: learned); + var forward = new InstrumentMaster(); + var reverse = new InstrumentMaster(); + + forward.Amend(first); + forward.Amend(second); + reverse.Amend(second); + reverse.Amend(first); + + Assert.Equal(forward.History(1), reverse.History(1)); + Assert.Equal("NEW", reverse.AsOf(1, new DateTime(2021, 2, 1))?.Symbol); + } + + [Fact] + public void AmbiguousBitemporalCoordinatesAreRejected() + { + var master = new InstrumentMaster(); + var record = Record(1, "A", Y2020, DateTime.MaxValue, recorded: Y2020); + master.Amend(record); + + Assert.Throws(() => master.Amend(record with { Symbol = "B" })); + } + /// Symbols are recycled, so a symbol alone does not identify an instrument. [Fact] public void ARecycledSymbolResolvesToWhicheverInstrumentHeldItThen() @@ -381,20 +490,23 @@ public void GapsAndOverlapsAreReported() withGap.Amend(Record(1, "B", new DateTime(2020, 7, 1), DateTime.MaxValue)); Assert.Contains(withGap.Validate(1), problem => problem.Contains("nothing covers")); - // Amend closes an interval it supersedes, so an in-order load cannot produce an - // overlap - the later record truncates the earlier one. var inOrder = new InstrumentMaster(); inOrder.Amend(Record(1, "A", Y2020, new DateTime(2020, 8, 1))); inOrder.Amend(Record(1, "B", new DateTime(2020, 7, 1), DateTime.MaxValue)); Assert.Empty(inOrder.Validate(1)); - // Out-of-order loading is the case that does, because Amend only closes forwards: - // the record already present starts later, so there is nothing to truncate. Loading a - // reference file in arbitrary order is ordinary, which is why Validate exists. var outOfOrder = new InstrumentMaster(); outOfOrder.Amend(Record(1, "B", new DateTime(2020, 7, 1), DateTime.MaxValue)); outOfOrder.Amend(Record(1, "A", Y2020, new DateTime(2020, 8, 1))); - Assert.Contains(outOfOrder.Validate(1), problem => problem.Contains("runs to")); + Assert.Empty(outOfOrder.Validate(1)); + + var retroactiveOverlap = new InstrumentMaster(); + retroactiveOverlap.Amend(Record(1, "B", new DateTime(2020, 7, 1), + DateTime.MaxValue)); + retroactiveOverlap.Amend(Record(1, "A", Y2020, new DateTime(2020, 8, 1), + recorded: new DateTime(2021, 1, 1))); + Assert.Contains(retroactiveOverlap.Validate(1), + problem => problem.Contains("runs to")); Assert.Contains(new InstrumentMaster().Validate(99), problem => problem.Contains("no records")); } @@ -450,9 +562,7 @@ public void AHaltOverridesTheSchedule() Assert.Equal(SessionState.Continuous, calendar.StateAt(from.AddMinutes(31), instrumentId: 7)); } - /// - /// The guard that stops an auction or a halt being read as a tradeable quote. - /// + /// Only continuous state produces a tradeable quote. [Fact] public void OnlyContinuousTradingCountsAsAQuote() { @@ -502,5 +612,41 @@ public void OverlappingWindowsAreRejected() new SessionWindow(SessionState.Continuous, new TimeSpan(9, 0, 0), new TimeSpan(12, 0, 0)), new SessionWindow(SessionState.PostClose, new TimeSpan(11, 0, 0), new TimeSpan(14, 0, 0)), })); + + [Fact] + public void InvalidWindowsAndHaltsAreRejected() + { + Assert.Throws(() => new SessionCalendar("BAD", + Array.Empty())); + Assert.Throws(() => new SessionCalendar("BAD", new[] + { + new SessionWindow(SessionState.Continuous, TimeSpan.FromHours(10), + TimeSpan.FromHours(9)), + })); + + var calendar = SessionCalendar.UsEquities(); + Assert.Throws(() => calendar.AddHalt(new TradingHalt(0, + Weekday, Weekday.AddMinutes(1), "invalid"))); + } + + [Fact] + public void ClosedSessionsTakePrecedenceOverHalts() + { + var saturday = new DateTime(2026, 8, 22, 10, 0, 0); + var calendar = SessionCalendar.UsEquities(); + calendar.AddHalt(new TradingHalt(7, saturday.AddHours(-1), saturday.AddHours(1), + "news pending")); + + Assert.Equal(SessionState.Closed, calendar.StateAt(saturday, instrumentId: 7)); + + var scheduledBreak = new SessionCalendar("TEST", new[] + { + new SessionWindow(SessionState.Closed, TimeSpan.FromHours(9), TimeSpan.FromHours(11)), + }); + scheduledBreak.AddHalt(new TradingHalt(7, Weekday.AddHours(9), Weekday.AddHours(11), + "news pending")); + Assert.Equal(SessionState.Closed, + scheduledBreak.StateAt(Weekday.AddHours(10), instrumentId: 7)); + } } } diff --git a/bench/docgen.py b/bench/docgen.py index ea3f589..271c08f 100755 --- a/bench/docgen.py +++ b/bench/docgen.py @@ -1,27 +1,5 @@ #!/usr/bin/env python3 -""" -Writes the generated tables in README.md and BENCHMARKS.md from bench/results/, -and verifies in CI that they still match. - -Why this exists: an audit found a published benchmark figure that had drifted -from its results file by roughly six times, in the direction that flattered the -argument the prose around it was making. Correcting that number once fixes one -number. Removing the hand that copies numbers fixes the class. - -Each generated region in a document is delimited: - - - ...table... - - -`--write` replaces the contents of every region from the results files. -`--check` exits non-zero if any region is stale, and prints the diff. Prose -outside the markers is written by hand and never touched. - -Usage: - python3 bench/docgen.py --write - python3 bench/docgen.py --check -""" +"""Generate benchmark tables from committed JSON; --check rejects drift.""" import argparse import difflib import glob @@ -219,6 +197,46 @@ def v2_protocol(): return table(["Case", "Packet", "Median", "Min–max", "Rate", "Allocation"], rows) +def v2_durability(): + data = load("durability-v2.json") + append = [] + for row in data["Append"]: + append.append([ + row["Name"], row["Policy"], f"{row['PayloadBytes']:,} B", + f"{row['MedianNanoseconds']:,.1f} ns", + f"{row['MinNanoseconds']:,.1f}–{row['MaxNanoseconds']:,.1f} ns", + f"{row['AppendsPerSecond']:,.0f}/s", f"{row['MedianSyncs']:,}", + f"{row['BytesAllocatedPerAppend']:.0f} B/op", + ]) + + recovery = data["Recovery"] + recovery_table = table( + ["Messages", "Checkpoint", "Full replay", "Checkpoint + tail", "Speed-up"], + [[f"{recovery['Messages']:,}", f"{recovery['CheckpointSequence']:,}", + f"{recovery['FullMedianMilliseconds']:.2f} ms " + f"({recovery['FullMinMilliseconds']:.2f}–{recovery['FullMaxMilliseconds']:.2f})", + f"{recovery['CheckpointMedianMilliseconds']:.2f} ms " + f"({recovery['CheckpointMinMilliseconds']:.2f}–" + f"{recovery['CheckpointMaxMilliseconds']:.2f})", + f"{recovery['SpeedUp']:.2f}×"]]) + + ranges = [] + for row in data["RangeReads"]: + ranges.append([ + row["Name"], f"{row['Queries']:,}", f"{row['IndexEntries']:,}", + f"{row['MedianNanosecondsPerRequest'] / 1000:,.1f} µs", + f"{row['MinNanosecondsPerRequest'] / 1000:,.1f}–" + f"{row['MaxNanosecondsPerRequest'] / 1000:,.1f} µs", + f"{row['BytesAllocatedPerRequest']:,.0f} B/request", + ]) + + return (table(["Append contract", "Policy", "Payload", "Median", "Min–max", + "Rate", "Syncs/trial", "Allocation"], append) + + "\n\n" + recovery_table + + "\n\n" + table(["10-message range", "Queries", "Index entries", "Median", + "Min–max", "Allocation"], ranges)) + + def v2_matching(): rows = [] for row in load("matching-v2.json")["Results"]: @@ -300,6 +318,8 @@ def v2_headline(): transitions += data["Transitions"][0]["RowsCompared"] exact = exact and all(r["RowsMatched"] == r["RowsCompared"] for r in data["Transitions"]) multicast = max(load("protocolv2-summary.json"), key=lambda r: r["Subscribers"]) + journal = next(r for r in load("durability-v2.json")["Append"] + if r["Name"] == "seal + packet WAL") rows = [ ["Feed encode → apply", f"{protocol['MedianNanoseconds']:.1f} ns median", f"{protocol['BytesAllocatedPerOperation']:.0f} B/op"], @@ -309,6 +329,8 @@ def v2_headline(): f"{measurement(matching_row['MatchReplenishCycleNs']):.1f} ns/cycle median", "state preserving"], ["Committed NASDAQ samples", f"{transitions:,} transitions per implementation", "exact" if exact else "mismatch present"], + ["Seal + journal feed packet", f"{journal['MedianNanoseconds']:,.1f} ns median", + f"{journal['BytesAllocatedPerAppend']:.0f} B/op; OS-buffered acknowledgement"], [f"Loopback multicast, {multicast['Subscribers']:,} subscribers", f"{multicast['MessagesPerSecond']:,.0f} delivered msg/s", f"{multicast['Gaps']} gaps; {multicast['IntegrityFailures']} CRC failures"], @@ -534,18 +556,7 @@ def head_to_head(): if not rows: raise MissingResults("no subscriber count was measured on both transports") - measured_together = {r[0] for r in rows} - only_multicast = [f"{r['Subscribers']:,}" for r in load_sweep("mcast") - if f"{r['Subscribers']:,}" not in measured_together] - - rendered = table(["Subscribers", "Unicast mean", "Multicast mean", "Improvement"], rows) - - if only_multicast: - rendered += ("\n\nMulticast was also measured at " - + ", ".join(only_multicast) - + " subscribers, where unicast was not run; those points are in the " - "multicast sweep in BENCHMARKS.md.") - return rendered + return table(["Subscribers", "Unicast mean", "Multicast mean", "Improvement"], rows) def equal_work(): @@ -622,17 +633,8 @@ def micros(row): f"**{mcast_cost:.2f} µs**" if mcast_cost else "—"], ] - rendered = table(["Transport", "Highest sustained subscribers", "Messages/s", "Server CPU", - "Server CPU per message"], rows) - - if unicast_cost and mcast_cost: - rendered += (f"\n\nMulticast delivers each message for " - f"**{unicast_cost / mcast_cost:.0f}× less server CPU**, to " - f"**{best_mcast['Subscribers'] / best_unicast['RequestedSubscribers']:.1f}× " - f"the subscribers** at " - f"**{best_mcast['MessagesPerSecond'] / best_unicast['MessagesPerSecond']:.1f}× " - f"the throughput**.") - return rendered + return table(["Transport", "Highest sustained subscribers", "Messages/s", "Server CPU", + "Server CPU per message"], rows) def headline(): @@ -716,6 +718,7 @@ def repeatability(): REGIONS = { "v2-environment": v2_environment, "v2-protocol": v2_protocol, + "v2-durability": v2_durability, "v2-matching": v2_matching, "v2-books": v2_books, "v2-queue": v2_queue, diff --git a/bench/results/durability-v2.json b/bench/results/durability-v2.json new file mode 100644 index 0000000..5d414a3 --- /dev/null +++ b/bench/results/durability-v2.json @@ -0,0 +1,116 @@ +{ + "TimestampUtc": "2026-08-19T19:30:05.8342953+00:00", + "Runtime": ".NET 8.0.30", + "OperatingSystem": "Ubuntu 24.04.3 LTS", + "Architecture": "X64", + "LogicalProcessors": 8, + "ServerGc": true, + "StopwatchFrequency": 1000000000, + "Crc32CImplementation": "SSE4.2", + "Records": 5000, + "PayloadBytes": 64, + "Trials": 5, + "Append": [ + { + "Name": "OS page cache", + "Policy": "OsBuffered", + "PayloadBytes": 64, + "Records": 5000, + "SyncIntervalMilliseconds": 200, + "MedianSyncs": 0, + "MedianNanoseconds": 976.74, + "MinNanoseconds": 958.01, + "MaxNanoseconds": 2088.2, + "AppendsPerSecond": 1023815, + "MegabytesPerSecond": 97.64, + "BytesAllocatedPerAppend": 0 + }, + { + "Name": "periodic 1 ms", + "Policy": "SyncPeriodic", + "PayloadBytes": 64, + "Records": 5000, + "SyncIntervalMilliseconds": 1, + "MedianSyncs": 4, + "MedianNanoseconds": 2318.34, + "MinNanoseconds": 2149.46, + "MaxNanoseconds": 3507.92, + "AppendsPerSecond": 431343, + "MegabytesPerSecond": 41.14, + "BytesAllocatedPerAppend": 0 + }, + { + "Name": "group commit 64", + "Policy": "OsBuffered", + "PayloadBytes": 64, + "Records": 5000, + "SyncIntervalMilliseconds": 200, + "MedianSyncs": 79, + "MedianNanoseconds": 27301.73, + "MinNanoseconds": 17837.92, + "MaxNanoseconds": 42412.73, + "AppendsPerSecond": 36628, + "MegabytesPerSecond": 3.49, + "BytesAllocatedPerAppend": 0 + }, + { + "Name": "fsync each", + "Policy": "SyncEachRecord", + "PayloadBytes": 64, + "Records": 5000, + "SyncIntervalMilliseconds": 200, + "MedianSyncs": 5000, + "MedianNanoseconds": 972646.2, + "MinNanoseconds": 898242.37, + "MaxNanoseconds": 1196500.72, + "AppendsPerSecond": 1028, + "MegabytesPerSecond": 0.1, + "BytesAllocatedPerAppend": 0 + }, + { + "Name": "seal \u002B packet WAL", + "Policy": "OsBuffered", + "PayloadBytes": 50, + "Records": 5000, + "SyncIntervalMilliseconds": 0, + "MedianSyncs": 0, + "MedianNanoseconds": 813.74, + "MinNanoseconds": 755.14, + "MaxNanoseconds": 1676.61, + "AppendsPerSecond": 1228894, + "MegabytesPerSecond": 100.79, + "BytesAllocatedPerAppend": 0 + } + ], + "Recovery": { + "Messages": 50000, + "CheckpointSequence": 47500, + "FullMedianMilliseconds": 24.68, + "FullMinMilliseconds": 21, + "FullMaxMilliseconds": 40.83, + "CheckpointMedianMilliseconds": 2.92, + "CheckpointMinMilliseconds": 2.64, + "CheckpointMaxMilliseconds": 3.84, + "SpeedUp": 8.45 + }, + "RangeReads": [ + { + "Name": "sparse index", + "Queries": 100, + "IndexEntries": 196, + "MedianNanosecondsPerRequest": 73567.12, + "MinNanosecondsPerRequest": 70164.95, + "MaxNanosecondsPerRequest": 81713.49, + "BytesAllocatedPerRequest": 1736 + }, + { + "Name": "segment scan", + "Queries": 100, + "IndexEntries": 0, + "MedianNanosecondsPerRequest": 1946832.33, + "MinNanosecondsPerRequest": 927098.33, + "MaxNanosecondsPerRequest": 2530302.62, + "BytesAllocatedPerRequest": 5848 + } + ] +} diff --git a/bench/results/durability.json b/bench/results/durability.json deleted file mode 100644 index 214d952..0000000 --- a/bench/results/durability.json +++ /dev/null @@ -1,56 +0,0 @@ -[ - { - "Kind": "append", - "Policy": "OsBuffered", - "PayloadBytes": 64, - "NanosecondsPerAppend": 276.2, - "AppendsPerSecond": 3620477, - "Syncs": 0, - "RunMilliseconds": 8.3, - "FlushIntervalMilliseconds": 200, - "MegabytesPerSecond": 345.3 - }, - { - "Kind": "append", - "Policy": "SyncPeriodic", - "PayloadBytes": 64, - "NanosecondsPerAppend": 325.4, - "AppendsPerSecond": 3072889, - "Syncs": 0, - "RunMilliseconds": 9.8, - "FlushIntervalMilliseconds": 200, - "MegabytesPerSecond": 293.1 - }, - { - "Kind": "append", - "Policy": "SyncEachRecord", - "PayloadBytes": 64, - "NanosecondsPerAppend": 182273.6, - "AppendsPerSecond": 5486, - "Syncs": 30000, - "RunMilliseconds": 5468.2, - "FlushIntervalMilliseconds": 200, - "MegabytesPerSecond": 0.5 - }, - { - "Kind": "recovery", - "Messages": 10000, - "FullReplayMilliseconds": 8.8, - "FromCheckpointMilliseconds": 1.9, - "SpeedUp": 4.6 - }, - { - "Kind": "recovery", - "Messages": 50000, - "FullReplayMilliseconds": 10.5, - "FromCheckpointMilliseconds": 1.9, - "SpeedUp": 5.4 - }, - { - "Kind": "recovery", - "Messages": 200000, - "FullReplayMilliseconds": 26.7, - "FromCheckpointMilliseconds": 5, - "SpeedUp": 5.3 - } -] \ No newline at end of file diff --git a/global.json b/global.json new file mode 100644 index 0000000..4c4c3ae --- /dev/null +++ b/global.json @@ -0,0 +1,7 @@ +{ + "sdk": { + "version": "8.0.100", + "rollForward": "latestFeature", + "allowPrerelease": false + } +} diff --git a/scripts/smoke.sh b/scripts/smoke.sh index fa60a3f..d9f1410 100755 --- a/scripts/smoke.sh +++ b/scripts/smoke.sh @@ -49,7 +49,10 @@ cat > "$WORK/multicast.json" <