Reference for tuning EventBroker(mt) and RequestBroker(mt) capacity,
payload sizing, and idle memory footprint.
See also: MT_BROKER_REFACTOR_RETROSPECTIVE.md §8 for the design rationale and the memory-mitigation heuristics this document references.
If you don't need to think about it: don't. The macros pick reasonable
defaults from your broker's type shape. The compile-time hint line
will tell you what was chosen and how much RAM the broker reserves at
init.
When you do need to tune — bursty workloads, large payloads, embedded targets — there are three composable mechanisms:
| Mechanism | When to reach for it |
|---|---|
preset = <name> |
one of the named profiles fits your case |
<knob> = <value> kwargs |
preset close enough, but a field needs override |
| neither | default sizing — the type-driven heuristic does the work |
All three may be combined. Resolution order is: defaults → preset → kwargs → type-driven auto for unset payload size fields.
emitter thread listener thread
────────────── ────────────────
emit(evt) poll-loop
│ │
▼ ▼
① alloc cell ─► ┌────────────────────┐ ⑤ deque idx
free-list │ Vyukov MPSC ring │ │
pop, no syscall│ (queueDepth slots)│ ▼
│ └────────────────────┘ ⑥ decode payload
▼ │ │
② mt_codec encodes │ ▼
into cell.payload[] │ ⑦ refcount--
│ │ │ if 0:
▼ ▼ ▼
③ refcount = N_listeners┌────────────────┐ cell back to slab
│ │ PayloadSlab │
▼ │ (slabCapacity │
④ enqueue cell idx ────►│ cells of │
│ maxPayload │
│ bytes each) │
└────────────────┘
RequestBroker only: ResponseSlotPool — `responseSlots` cells of
`maxResponseBytes` payload. Reserved at request
issue, freed on caller-side decode.
Each macro call reserves all three structures up-front (per (broker, context, listener-thread)). No allocation on the hot path.
| Knob | Default | Drives | Drop trigger | RAM cost |
|---|---|---|---|---|
queueDepth |
256 | Ring slots (must be power-of-2) | bursty emit briefly outpacing the listener → ring full | ~24 B/slot |
slabCapacity |
1024 | Total cells = concurrent payloads alive (cell lifetime ≈ cross-thread hop + listener async work) | slow listener holds many cells → slab exhausted | 1 × cellStride per cell |
maxPayloadBytes |
type-driven | Per-cell payload buffer (cellStride = sizeof(header) + maxPayloadBytes, aligned to 8) |
marshaled event too large for the cell | direct multiplier on slab RAM |
freeListShards |
4 | Slab free-list shards. More = less CAS contention between concurrent emitters | almost never matters below 16 threads | ~16 B per shard |
Total event-broker idle RAM ≈
queueDepth × 24 + slabCapacity × align8(headerBytes + maxPayloadBytes).
| Knob | Default | Drives | Failure mode | RAM cost |
|---|---|---|---|---|
queueDepth |
256 | Ring slots for request messages to the provider | bursty requests outpace provider | ~24 B/slot |
slabCapacity |
64 | Cells for in-flight request payloads (lifetime ≈ cross-thread hop only — the provider releases the cell as soon as it has decoded the args, before doing the work) | highly concurrent fan-out from caller | 1 × cellStride per cell |
maxPayloadBytes |
type-driven (proc params) | Per-request marshal buffer | large request args → encode fails → err |
multiplier on slab RAM |
responseSlots |
256 | Concurrent outstanding requests (slot held caller-side from request issue until reply is decoded — RTT-bound). | more than responseSlots in-flight at once |
1 × slotStride per slot |
maxResponseBytes |
type-driven (broker type) | Per-response marshal buffer | large reply → encode fails | multiplier on responseSlots RAM |
freeListShards |
2 | Same as event | rarely matters | ~16 B per shard |
Total request-broker idle RAM ≈
queueDepth × 24 + slabCapacity × align8(headerBytes + maxPayloadBytes) + responseSlots × align8(slotHeaderBytes + maxResponseBytes).
| Knob | Rough rule |
|---|---|
queueDepth |
expected_burst_rate × hop_time_µs / 1_000_000, rounded up to power-of-2. Cheap to over-provision (24 B/slot). |
slabCapacity |
similar — recycle is fast (microseconds), so even bursty workloads do fine with O(1000). |
responseSlots |
matches your concurrent-in-flight ceiling. RTT-bound, slow to recycle. |
maxPayloadBytes / maxResponseBytes |
upper bound on a single marshaled payload. Leave on default and let the type classifier size it. |
freeListShards |
leave alone unless profiling shows CAS contention. |
When maxPayloadBytes / maxResponseBytes is left at its default the
macro inspects the type AST at compile time and picks a size class.
The classifier walks the type recursively — an Option[T] recurses
into T, an array[N, T] recurses into T — so the bucket reported
in the compile-time hint always reflects the worst-case wire-bound
payload, never an arbitrary outer-bracket fallback.
| Type shape | Auto cell size | Origin tag emitted |
|---|---|---|
scalar (bool, int*, uint*, float*, byte, char) |
64 B | auto:scalar (object body) / auto:scalar:<name> (bare type) |
string |
4 KB | auto:string |
seq[byte] / seq[uint8] |
64 KB | auto:seq[byte] |
seq[string] |
16 KB | auto:seq[string] |
seq[<other primitive>] |
4 KB | auto:seq[<T>] |
array[N, T] |
bucket of T (recursive) |
inherits T's origin |
Option[T] |
bucket of T (recursive) |
auto:Option[<inner-reason>] |
Option[seq[byte]] |
64 KB (= inner seq[byte]) |
auto:Option[seq[byte]] |
Option[scalar] |
64 B | auto:Option[scalar:<name>] |
void / zero-field body |
64 B (envelope only) | auto:void |
inline object |
max over classified fields | inherits dominant field's origin |
| alias / external / unresolvable | 8 KB + compile-time warning | auto:unclassifiable:<repr> |
For an object type the cell is sized to fit the largest classified
field (classifyFieldsMax).
void-bodied brokers — typed as type Foo = void for an event, or a
zero-arg proc signature*(): Future[Result[Foo, string]] paired with
type Foo = void for a request — collapse to the scalar bucket
(64 B) rather than the conservative 1 KB / 64 KB safety default.
A payload-less notification only ships the CBOR envelope, so a
larger cell would pin idle slab/respPool memory for no reason. For a
single zero-field RequestBroker(mt): type X = void this is the
difference between 40 KB and 16 MB idle RAM per context.
Option[T] recurses into T rather than falling into the
unclassifiable bucket. This is both more accurate and safer:
without the recursion Option[seq[byte]] silently under-allocated
to 8 KB while its inner seq[byte] can carry 64 KB, which would
surface as a runtime cell-too-small error rather than a clean
compile-time hint.
When the classifier emits a warning it means it could not introspect
the type (typically because it's an alias to a non-primitive, or a
forward-declared external type). The build still succeeds with an
8 KB fallback, but you should provide maxPayloadBytes = N /
maxResponseBytes = N explicitly so the size matches your actual
payload. The fix is usually one line:
# `Address` is an external object — classifier can't introspect it
EventBroker(mt, maxPayloadBytes = 256):
type Connected = object
peer*: AddressBuilt-in profiles for the most common shapes. Selected via
preset = <name> in the macro kwargs.
| Preset | EventBroker(mt) | RequestBroker(mt) |
|---|---|---|
defaultBalanced |
ring=256, slab=1024, payload=1 KB, shards=4 | ring=256, slab=64, payload=1 KB, respSlots=256, respBytes=64 KB, shards=2 |
fastBurst |
ring=4096, slab=8192, payload=256, shards=8 | ring=4096, slab=256, payload=256, respSlots=1024, respBytes=4 KB, shards=4 |
largePayload |
ring=64, slab=128, payload=64 KB, shards=2 | ring=64, slab=32, payload=64 KB, respSlots=64, respBytes=256 KB, shards=2 |
tinyFootprint |
ring=32, slab=32, payload=256, shards=1 | ring=16, slab=8, payload=256, respSlots=16, respBytes=1024, shards=1 |
Individual kwargs supplied alongside the preset override the preset's fields:
EventBroker(mt, preset = fastBurst, maxPayloadBytes = 1024):
type MyEvt = ...User-defined presets (passing an external const struct) are planned but not yet implemented. The kwarg parser emits a clear error if you try to pass a non-built-in preset name.
Every MT broker macro callsite emits a hint line. Example for the
event-broker performance test:
Hint: [brokers] EventBroker(PerfEvt): queueDepth=4096 [preset:fastBurst],
slabCapacity=8192 [preset:fastBurst],
maxPayloadBytes=1024 [kwarg],
freeListShards=8 [preset:fastBurst]
— idle RAM: ring≈96.0 KB, slab≈8.2 MB, total≈8.3 MB
The provenance tag [origin] per field:
| Tag | Meaning |
|---|---|
default |
unchanged from the module's default |
kwarg |
explicit <knob> = <value> in the macro call |
preset:<name> |
inherited from a preset = <name> kwarg |
auto:<reason> |
derived by the type-driven default. Examples: auto:scalar (object body whose biggest field is a scalar) / auto:scalar:int64 (bare scalar type at classifier root) / auto:string / auto:seq[byte] / auto:seq[string] |
auto:void |
broker body is type X = void or has zero fields — cell collapsed to the 64 B scalar bucket (envelope only) |
auto:Option[<inner-reason>] |
Option[T] — recursed into T and inherited its bucket (e.g. auto:Option[seq[byte]] = 64 KB, auto:Option[scalar:int64] = 64 B) |
auto:unclassifiable:<...> |
fallback for a type the classifier couldn't resolve — pair with a compile-time {.warning.} |
Idle RAM is an estimate (per-element bytes are approximate and exclude
the small fixed overhead of the broker's bucket / global lock). Use it
to compare configurations and catch surprises — the exact value depends
on alignment and may differ by a few percent from top.
To silence the line per build: -d:brokerConfigSilent.
To silence the entire User category: --hint[User]:off.
EventBroker(mt):
type Tick = object
timestampNs*: int64
seqNum*: uint32
# Hint: [brokers] EventBroker(Tick): queueDepth=256 [default],
# slabCapacity=1024 [default], maxPayloadBytes=64 [auto:scalar],
# freeListShards=4 [default]
# — idle RAM: ring≈6.0 KB, slab≈92.0 KB, total≈98.0 KBEventBroker(mt):
type Heartbeat = void
# Hint: [brokers] EventBroker(Heartbeat): … maxPayloadBytes=64 [auto:void] …
# — idle RAM: total≈102.0 KBThe same shape applies to a notification-only request:
RequestBroker(mt):
type Ack = void
proc signature*(): Future[Result[Ack, string]] {.async.}
# Hint: [brokers] RequestBroker(Ack): maxPayloadBytes=64 [auto:void],
# maxResponseBytes=64 [auto:void] … — idle RAM: total≈40.0 KBWithout the auto:void collapse this would pin 16 MB for the response
pool of every notification-style RequestBroker.
import std/options
EventBroker(mt):
type ChunkReady = object
blob*: Option[seq[byte]] # → auto:Option[seq[byte]] = 64 KB
label*: string # → auto:string = 4 KB
# The object bucket = max(64 KB, 4 KB) = 64 KB.
# Hint: maxPayloadBytes=65536 [auto:Option[seq[byte]]]Compare to before the fix: Option[seq[byte]] fell into the
unclassifiable arm at 8 KB — too small for the wire-bound 64 KB,
so a large emit would have returned err(...) at runtime. The
recursive Option[T] classification eliminates that failure mode.
EventBroker(mt, preset = fastBurst, maxPayloadBytes = 1024):
type WireEvent = object
payload*: seq[byte]RequestBroker(mt, preset = largePayload, responseSlots = 32):
type FetchResult = object
blob*: seq[byte]
proc fetch*(key: string): Future[Result[FetchResult, string]] {.async.}EventBroker(mt, preset = tinyFootprint):
type LedToggle = object
on*: bool| Symptom | Likely cause | Fix |
|---|---|---|
Chronicles WRN event dropped: listener queue full |
Burst emit rate > listener drain rate over the ring's depth | bump queueDepth and/or slabCapacity; or pick preset = fastBurst |
RequestBroker(...): provider queue full error returned |
Same as above on the request path | bump queueDepth, or throttle issue rate |
RequestBroker(...): no response slot available error |
More concurrent in-flight requests than responseSlots |
bump responseSlots to match your concurrency ceiling |
Compile-time unclassifiable:<name> warning |
Type classifier couldn't introspect an alias / external type | provide explicit maxPayloadBytes = N / maxResponseBytes = N |
| Idle RAM higher than expected | Auto-classified large field (e.g. seq[byte] → 64 KB cells × 1024 slab) |
use preset = largePayload (narrows slab) or explicit slabCapacity |
A future optimization for brokers whose entire payload fits in a single ring slot — typically the "tick / heartbeat / counter / flag" pattern that makes up an estimated 40–50 % of real-world MT brokers.
For event types whose payload is one scalar field (≤ 16 B — int64,
uint32, bool, enum, distinct of a scalar), the value would live
directly in the ring slot, with no payload slab and no marshaler
involvement.
| Step | Today (slab path) | Tier-A (inline) |
|---|---|---|
| Send | alloc slab cell → marshal into cell → enqueue cellIdx: uint32 → bump refcount |
tryEnqueue value into ring |
| Receive | dequeue idx → load cell → unmarshal → call handler → decRef → release to slab | tryDequeue value → call handler |
| Idle RAM | ring + slab (~100 KB at defaults) | ring only (~6 KB at defaults) |
| Per-event cost | ~5 atomics + 2 memcpy + slab claim/release | 1 enqueue + 1 dequeue |
The VyukovMpscRing[T] primitive in mt_queue.nim is
already generic over T, so the queue itself supports Tier-A. The
implementation cost is in the broker generator, which today is
~700 LOC tightly coupled to the slab/refcount model:
- The
CtrlClearListeners = high(uint32) - 1sentinel is multiplexed onto the same ring as event payloads (mt_event_broker.nim:50). When the ring'sTis the event type instead ofuint32, the sentinel must move to a separateAtomic[bool] needsClearchannel. - Emit / listener / drop code paths all assume cell allocation, marshal/unmarshal, and refcounted release; each would need a parallel inline-mode body.
Cleanest approach: a parallel generateMtEventBrokerTierA(body, cfg)
generator (~350–450 LOC), with the existing macro dispatching to it
when cfg.tierA is set.
At macro time, set cfg.tierA = true when:
- The broker type is an
objectwith exactly one scalar field, or - The broker type is a
distinct/ alias of a scalar, or - (Optional, broader) The broker type is a small object whose total size is ≤ 16 B and every field is scalar.
When detected, also force slabCapacity / maxPayloadBytes / related
slab-only knobs to be moot (and surface that in the compile-time hint
as [tier-a:inline]).
For an event broker carrying a single int64:
| Defaults today | Tier-A | |
|---|---|---|
| Idle RAM | ~98 KB | ~6 KB (16×) |
| Per-event allocator interaction | slab claim + release | none |
| Per-event atomics | ~5 | ~2 |
Cross-thread throughput would benefit by an estimated 2–3× on purely-scalar brokers (no measurement yet — predictive based on removing the slab claim / refcount round-trip from the hot path).
Deferred. The detection + flag wiring is a small follow-on; the generator work is a dedicated mini-project of its own and should be scoped + reviewed separately. No timeline committed.