[oscarlius]

What this changes

Concurrent agents interleave conversations while waiting on tools or other tasks. Once the active contexts and count-capped slots are exhausted, returning to an unchanged conversation can require reading its prompt again despite spare host memory and disk capacity.

Add opt-in RAM-to-disk conversation slots: keep bounded hot states in RAM, spill the RAM LRU, and trim disk states by actual last use. Preserve recency across spill/restart, reject oversized/zero-budget writes before evicting other files, flush normal CLI/serve returns, and distinguish miss reasons without logging prompt text.

MAYA_GLM_SLOT_SPILL remains off by default. Both budgets are configurable: MAYA_GLM_SLOT_RAM_GB falls back to 8 GiB when opted in; disk uses the existing MAYA_GLM_SLOT_GB fallback of 16 GiB. No deployment-specific allocation is embedded. MAYA_GLM_SLOTS=0 still disables slots.

This branch is based on current main and excludes the development branch's separate NUMA/shared-pool and dynamic-DFlash changes. It changes conversation retention, not the number of decoding workers or the scheduler.

Answers

• Exact PR-head model-token parity was rerun.
• Tiering is opt-in; unset/zero keeps the existing storage/count policies, covered by host regression.

No weights, routing, sampling or prefix/image compatibility rules are changed. Full real-model disk restoration and token-parity validation remain outside the evidence below.

Tested on

• This standalone branch: Windows, MSVC 14.50, C++20. 17/17 source-extracted host scenarios passed, including the two default-setting cases; complete modified generate.cpp compiled as a host object. No CUDA link or model execution on this branch. The portable harness is wired into CPU-only CI.
• Existing repaired deployment, not this PR head: v1.1.63 development build, GLM-5.3-Flash Maya-L, four RTX 4090s at 255W, about 251.3GiB RAM, 144 CPU lanes, 4 contexts, 1M context limit, prefill 5120, split 12,24,35. Linux/CUDA incremental link passed. Seven bounded model requests returned OK; returning to a 1470-token prompt restored 1469 tokens from RAM, while an edited beginning restored 0. Natural traffic interleaved.
• CI for 082464a: upstream Linux Python/host checks and macOS/Metal checks passed. The new conversation-slot step passed on Linux; the fork push workflows also passed.
• Historical negative control: original spill prototype passed 6/15 host cases; repaired implementation passed 15/15. These were reused, not rerun.

Validation details and machine-readable receipts: scope and limits, standalone host results, historical results.

Speed

No new throughput or production-average claim. In an earlier pre-fix capacity observation, returning to the first of nine 9236-token conversations took 4.480s with legacy slots (0 cached tokens) versus 0.202s with RAM spill (9235 cached tokens). This was one synthetic return with interleaved traffic, not an isolated multi-agent benchmark. Five conversations had not exhausted the active contexts plus saved slots.

No new agent_bench.py run, decode/prefill A/B, full quality suite, full-budget pressure test, or real-model disk-overflow/restart test was performed. Historical RAM hits do not validate disk restoration. Preparing this PR did not connect to the model host, start WSL, send inference requests, or change the running service.

Checks

• Host fixture and full modified translation unit compile on the platform tested; current-branch CUDA link is untested.
• 17 host cases passed locally; CPU-only CI is configured to run the portable harness.
• Settings and validation boundaries documented; sanitized receipts committed.
• Independent read-only review found no P1/P2 issues; documentation links checked.