Found while building and measuring the failure knee for #74 (branch feat/knee-step4-the-knee, parked with full measurements).
The finding
compute, db, alb and waf have no maxQueueSize — only sqs (200), stream, gpu and infgw are capped. An unbounded queue has no stationary overload state:
- arrivals ≤ service rate → queue stays finite, failures ≈ 0
- arrivals > service rate → queue grows without limit, utilization runs away, failures → 100 %
The board is therefore bistable in arrival rate. Measured on the reference board (tests/helpers/reference-board.mjs): 6 rps gives 7 failures at reputation 95, and 8 rps gives 165 failures at reputation −43. There is no arrival rate in between where the board fails a little and keeps running.
This is why the failure-knee curve could not produce a readable middle at any calibration (onset 0.90 / 1.00 / 1.25 all measured 0 % band width, at bottleneck capacity 4 and capacity 10 alike). The failure curve was never the lever.
Why it matters
Issue #74 ("bored in 10 minutes") is about the missing middle between "fine" and "dead". A player gets no state they can read and act on, because the simulation has no such state to be in.
What would fix it
Bounded queues + load shedding on the compute family. Past the backlog limit the excess is dropped, utilization pins near 1.0, and the system runs indefinitely at a loss rate proportional to the overshoot — a genuine stationary state. That is:
- the mechanic the game already models correctly in SQS (
maxQueueSize: 200)
- a real cloud lesson (backpressure, shed load,
503 over unbounded latency) rather than a tuned probability
- the thing that makes a readable band physically possible
Needs care on: campaign balance (a queue cap changes shipped levels, so it must be survival-gated or the levels re-proven), the cardinal invariant (#191/#192 — shed requests must terminate exactly once), and the interaction with retries and the DLQ.
Related, found at the same time
failChance is computed after this.processing.splice(i, 1) in Service.update, so the load the curve reads is short by exactly one job at the moment of the roll — 0.25 of utilization on a capacity-4 Compute, 0.10 on a capacity-10 one. The effective failure threshold is therefore not "100 % of capacity" but 100 % plus a bias that shrinks as nodes get bigger. Filing separately would duplicate context; noting it here.
Found while building and measuring the failure knee for #74 (branch
feat/knee-step4-the-knee, parked with full measurements).The finding
compute,db,albandwafhave nomaxQueueSize— onlysqs(200),stream,gpuandinfgware capped. An unbounded queue has no stationary overload state:The board is therefore bistable in arrival rate. Measured on the reference board (
tests/helpers/reference-board.mjs): 6 rps gives 7 failures at reputation 95, and 8 rps gives 165 failures at reputation −43. There is no arrival rate in between where the board fails a little and keeps running.This is why the failure-knee curve could not produce a readable middle at any calibration (onset 0.90 / 1.00 / 1.25 all measured 0 % band width, at bottleneck capacity 4 and capacity 10 alike). The failure curve was never the lever.
Why it matters
Issue #74 ("bored in 10 minutes") is about the missing middle between "fine" and "dead". A player gets no state they can read and act on, because the simulation has no such state to be in.
What would fix it
Bounded queues + load shedding on the compute family. Past the backlog limit the excess is dropped, utilization pins near 1.0, and the system runs indefinitely at a loss rate proportional to the overshoot — a genuine stationary state. That is:
maxQueueSize: 200)503over unbounded latency) rather than a tuned probabilityNeeds care on: campaign balance (a queue cap changes shipped levels, so it must be survival-gated or the levels re-proven), the cardinal invariant (#191/#192 — shed requests must terminate exactly once), and the interaction with retries and the DLQ.
Related, found at the same time
failChanceis computed afterthis.processing.splice(i, 1)inService.update, so the load the curve reads is short by exactly one job at the moment of the roll — 0.25 of utilization on a capacity-4 Compute, 0.10 on a capacity-10 one. The effective failure threshold is therefore not "100 % of capacity" but 100 % plus a bias that shrinks as nodes get bigger. Filing separately would duplicate context; noting it here.