Skip to content

Commit ee76888

Browse files
committed
Optimize Databento client throughput
- Add end-to-end Databento historical and live client benches - Fast-path single live MBO deltas without buffer map churn - Route historical benches through local HTTP fixtures - Extend mock LSG writes for large MBO byte bursts - Cover single-delta MBO and custom base URL error paths
1 parent 49d2be1 commit ee76888

8 files changed

Lines changed: 637 additions & 6 deletions

File tree

crates/adapters/databento/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,10 @@ harness = false
103103
name = "micros"
104104
harness = false
105105

106+
[[bench]]
107+
name = "clients"
108+
harness = false
109+
106110
[[example]]
107111
name = "databento-data-tester"
108112
path = "examples/node_data_tester.rs"

crates/adapters/databento/benches/BENCHMARKS.md

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Absolute numbers vary by machine; only same-machine deltas are meaningful.
1313
```bash
1414
sudo cpupower frequency-set -g performance
1515
CARGO_BUILD_JOBS=16 setarch "$(uname -m)" -R cargo bench -p nautilus-databento --profile bench-lto \
16-
--bench data --bench micros
16+
--bench data --bench micros --bench clients
1717
sudo cpupower frequency-set -g powersave
1818
```
1919

@@ -74,6 +74,28 @@ data files.
7474
| `large_mbo/loader_collect` | 5.67 ms | 11.6 M/s |
7575
| `large_mbo/loader_stream_count` | 5.35 ms | 12.3 M/s |
7676

77+
## Client end-to-end (`clients.rs`)
78+
79+
Deterministic local-network benches for the public client surfaces. The
80+
historical rows route `DatabentoHistoricalClient` through a local HTTP fixture
81+
server that returns committed DBN zstd fixtures. The live row routes
82+
`DatabentoFeedHandler` through the mock LSG protocol server. The trade row
83+
measures one session with authenticate, subscribe, start, symbol mapping, 100
84+
trade records, message-channel receive, and close. The MBO row keeps one live
85+
session open after subscribe/start warmup, then measures repeated 10,000-record
86+
MBO byte bursts through socket read, DBN decode, live handler dispatch, MBO
87+
buffering, and message-channel receive.
88+
89+
These rows are useful for same-machine regressions in client orchestration.
90+
They are not external Databento service latency claims.
91+
92+
| Bench | Median | Throughput |
93+
|--------------------------------------|---------|------------|
94+
| `historical_client/trades_http` | 65.5 µs | 30.5 k/s |
95+
| `historical_client/mbp1_quotes_http` | 66.3 µs | 30.2 k/s |
96+
| `live_client/trades_mock_lsg` | 41.1 ms | 2.43 k/s |
97+
| `live_client/mbo_stream_mock_lsg` | 3.40 ms | 2.94 M/s |
98+
7799
## Component breakdown (`micros.rs`)
78100

79101
Diagnostic benches that decompose the pipeline numbers above. Use these to

0 commit comments

Comments
 (0)