fix(responses): bound the durable spill directory with an aggregate byte cap

The response store has an unconditional RAM ceiling
(MAX_STORED_RESPONSE_BYTES, 64 MiB) and demotes the oldest resident entry
to a durable spill once it is crossed. Nothing bounded where those bytes
landed: the spilled set was capped only per file
(MAX_RESPONSE_SPILL_PAYLOAD_BYTES, 256 MiB) and per entry
(MAX_STORED_RESPONSES, 1000). Their product is 250 GiB, larger than the
disk of any host this runs on, so the only effective bound was
RESPONSE_TTL_MS and disk use became a function of client request rate
rather than of anything this process controls.

Measured on one macOS host, 2026-08-30: a client spilling ~150 MB
payloads at ~1.4/min held 6.8 GB of ~/.opencodex/responses-state-spill
after 44 minutes and was still climbing toward the ~12 GB an hour-long
window implies. It filled the volume, at which point unrelated processes
began failing with ENOSPC. Retention itself was correct throughout - the
TTL evicted that whole cohort an hour later - so this is a missing
budget, not a leak.

Add MAX_SPILLED_RESPONSE_BYTES (1 GiB) and evict past it in
pruneResponses, immediately after the RAM demotion loop that creates the
pressure. deleteEntry already routes through deleteOwnedSpills, so an
evicted entry unlinks its file.

Eviction is ordered by createdAt, not by map order. `states` is not an
age index: demotion and spill replacement delete and reinsert entries,
and writeBoundedSnapshot serializes the map reversed, so map order can
put a newer continuation first and evicting that one spends a resume the
older entry would not have cost. createdAt is millisecond-resolution and
ties are ordinary under load, where a stable sort would fall back to
insertion order, so ties break on the response id - by direct comparison
rather than localeCompare, since the order must not depend on the host
locale.

The spilled total is recomputed per prune rather than carried as a
running counter: spilled entries reach `states` through several
insertion paths (demotion swap, direct oversized admission, snapshot
reload), and one missed increment there would silently disable the cap,
where a walk over at most MAX_STORED_RESPONSES entries cannot drift.

1 GiB comes from the same sample (n=31), whose spilled sizes are
strongly bimodal: median 1.1 MiB against a p90 of 198.7 MiB. At that
median the count cap and this ceiling bind within 8% of each other
(1000 x 1.1 MiB = 1.07 GiB), so ordinary traffic sees no eviction it
would not already have seen and only the large tail is cut. The value is
the one knob here a maintainer may reasonably want to change.

The spill directory was documented nowhere, while the state-root
inventory describes its group as bounded caches, so structure/00 and
structure/02 now name it and its aggregate bound.

Each regression test was confirmed to fail without the code it covers:
the budget test retains 48564 bytes against a 20000-byte cap without the
eviction loop, the ordering test keeps the older spill without the
createdAt sort, and the tie test does the same without the id
tie-breaker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: lifrary

src/responses/state.ts

@@ -28,6 +28,30 @@ const SNAPSHOT_DEBOUNCE_MAX_MS = 30_000;
  * continuation chains) stores the full expanded input each turn — ~quadratic bytes per chain —
  * so a count cap alone cannot bound memory. Oldest-first eviction applies past this mark. */
 export const MAX_STORED_RESPONSE_BYTES = 64 * 1024 * 1024;
+/**
+ * Aggregate ceiling for the durable spill directory: the disk-side counterpart to
+ * the RAM ceiling above. Without it the spilled set is bounded only per-file
+ * (MAX_RESPONSE_SPILL_PAYLOAD_BYTES, 256 MiB) and per-entry (MAX_STORED_RESPONSES,
+ * 1000), whose product is 250 GiB — larger than the disk of any host this runs on.
+ * The only effective bound was therefore RESPONSE_TTL_MS, which makes disk use a
+ * function of client request rate rather than of anything this process controls.
+ *
+ * Measured on one macOS host, 2026-08-30: a client spilling ~150 MB payloads at
+ * ~1.4/min held 6.8 GB after 44 minutes, still climbing toward the ~12 GB an
+ * hour-long window implies, and filled the volume. Retention itself was correct
+ * throughout — the TTL evicted that whole cohort an hour later — so what was
+ * missing is a budget, not a sweep.
+ *
+ * 1 GiB comes from the same sample (n=31), whose spilled sizes are strongly
+ * bimodal: median 1.1 MiB against a p90 of 198.7 MiB, near the per-file ceiling.
+ * At that median the count cap and this ceiling bind within 8% of each other
+ * (1000 x 1.1 MiB = 1.07 GiB), so ordinary traffic sees no eviction it would not
+ * already have seen and only the large tail is cut. Erring small is the safe
+ * direction: too low costs a replay miss, an already-handled path surfaced as
+ * previous_response_not_found, while too high costs the host's disk and every
+ * unrelated process on it.
+ */
+export const MAX_SPILLED_RESPONSE_BYTES = 1024 * 1024 * 1024;
 /** Legacy snapshot selection only. Spill demotion is governed solely by the RAM cap above. */
 const SNAPSHOT_ENTRY_MAX_BYTES = 2 * 1024 * 1024;
 const SNAPSHOT_TOTAL_MAX_BYTES = 24 * 1024 * 1024;
@@ -174,6 +198,37 @@ export function getStoredResponseBytesForTests(): number {
   return storedResponseBytes;
 }
 
+let spillByteCapOverride: number | null = null;
+
+function spillByteCap(): number {
+  return spillByteCapOverride ?? MAX_SPILLED_RESPONSE_BYTES;
+}
+
+/**
+ * Live total of durable spill payloads. Recomputed per call rather than carried as
+ * a running counter: spilled entries reach `states` through several insertion paths
+ * (demotion swap, direct oversized admission, snapshot reload), and one missed
+ * increment there would silently disable the cap, where an O(MAX_STORED_RESPONSES)
+ * walk cannot drift.
+ */
+function spilledResponseBytes(): number {
+  let total = 0;
+  for (const entry of states.values()) {
+    if (entry.kind === "spill") total += entry.spill.payloadBytes;
+  }
+  return total;
+}
+
+/** Test-only: lower/restore the durable spill cap (null restores the default). */
+export function setSpilledResponseByteCapForTests(bytes: number | null): void {
+  spillByteCapOverride = bytes;
+}
+
+/** Test-only: current durable spill accounting (proves evictions unlink their files). */
+export function getSpilledResponseBytesForTests(): number {
+  return spilledResponseBytes();
+}
+
 function serializedBytes(value: unknown): number | null {
   try {
     const serialized = JSON.stringify(value);
@@ -1070,6 +1125,31 @@ function pruneResponses(at = now()): void {
       replaceWithSpillFailure(oldestId, entry);
     }
   }
+  // Disk counterpart to the RAM cap above. Demotion moves bytes out of memory but
+  // nothing bounded where they land, so evict oldest-first until the durable set
+  // fits. deleteEntry routes through deleteOwnedSpills, which unlinks the file.
+  let spilledBytes = spilledResponseBytes();
+  if (spilledBytes > spillByteCap()) {
+    // Ordered by createdAt, not by map order. `states` is not an age index:
+    // demotion and spill replacement delete and reinsert entries, and
+    // writeBoundedSnapshot serializes the map reversed, so map order can put a
+    // newer continuation first — and evicting that one spends a resume the older
+    // entry would not have cost. Sorting is O(k log k) over the spilled subset
+    // and runs only on a tick already over budget.
+    const spilled = [...states]
+      .filter((pair): pair is [string, SpilledResponseState] => pair[1].kind === "spill")
+      // createdAt is millisecond-resolution, so ties are ordinary under load. A
+      // stable sort would then fall back to insertion order — the very order this
+      // is avoiding — so break ties on the response id. Not localeCompare: the
+      // order must not depend on the host locale.
+      .sort((a, b) => a[1].createdAt - b[1].createdAt
+        || (a[0] < b[0] ? -1 : a[0] > b[0] ? 1 : 0));
+    for (const [id, entry] of spilled) {
+      if (spilledBytes <= spillByteCap()) break;
+      spilledBytes -= entry.spill.payloadBytes;
+      deleteEntry(id);
+    }
+  }
 }
 
 /** Periodic TTL-only sweep; count/byte eviction remains owned by mutation paths. */

structure/00_overview.md

@@ -84,7 +84,7 @@ opencodex state root does not undo those writes. Putting native Codex back is th
 | `~/.opencodex/ocx.pid`, `runtime-port.json`, `system-env-port` | opencodex runtime | Live process identity and the port a client should reach; rewritten on start. `runtime-port.json` also carries the protected per-process listener-attestation key used before CLI diagnostics attach a management bearer. |
 | `~/.opencodex/codex-runtime.json`, `codex-runtime-clamp.json` | opencodex Codex runtime | Selected Codex executable/version state and effort-clamp diagnostics. Not process identity: these persist a resolved choice and a diagnostic, so losing them changes behavior until re-resolved. |
 | `~/.opencodex/service-state.json`, `service.log`, `service-api-token`, `opencodex-service-launcher.vbs`, `opencodex-service-task.xml`, `opencodex-service.cmd`, `winsw`, `tray-state.json`, `tray-heartbeat.json`, `opencodex-tray.ps1`, `opencodex-tray-*.ico`, `update-job.json` | opencodex operators | Installed-service, Windows tray, and self-update artifacts and bookkeeping. The update record carries its worker PID so a dead worker recovers instead of blocking later runs. |
-| `~/.opencodex/responses-state.json`, `usage-debug.jsonl`, `crash.log`, `artifacts/` | opencodex diagnostics and artifacts | Bounded caches, diagnostics, and generated image/video artifacts served locally. |
+| `~/.opencodex/responses-state.json`, `responses-state-spill/`, `usage-debug.jsonl`, `crash.log`, `artifacts/` | opencodex diagnostics and artifacts | Bounded caches, diagnostics, and generated image/video artifacts served locally. The spill directory holds continuation state demoted out of the in-memory cap and is bounded in aggregate, not only per file. |
 | `~/.opencodex/codex-shim.json`, `*.lock`, `kimi-device-id`, `mimo-client-id`, `.star-prompted` | opencodex bookkeeping | Shim restore obligations, cross-process locks, per-install client identifiers, one-shot UI flags. |
 | `~/.opencodex/.opencodex-owner.json`, `.opencodex-uninstall.json` | opencodex | Ownership marker and the manifest that bounds what uninstall may remove. Both live in the OpenCodex state root, not in `$CODEX_HOME`. |
 | `$CODEX_HOME/config.toml` | Codex, edited by opencodex | Active provider and provider table. |

structure/02_config-and-codex-home.md

@@ -157,6 +157,23 @@ Hidden` inside an already-running PowerShell script, nor to .NET/VBS process-win
 - 다른 대안 대신 이 방식을 선택한 이유: Names and environment paths are caller-controlled, required secret writes must not silently skip ACLs, and elevation has a larger authority boundary that should remain FFI-only.
 - 장점, 단점 및 영향: Default Windows ARM64 installations can start and harden secrets; non-default Windows roots continue to fail closed until Bun exposes a trustworthy native system-directory API without FFI.
 
+The durable response-spill directory `~/.opencodex/responses-state-spill/` is bounded in
+aggregate, not only per file. Continuation state demoted out of the in-memory cap
+(`MAX_STORED_RESPONSE_BYTES`) is written there, and eviction past
+`MAX_SPILLED_RESPONSE_BYTES` removes oldest-first through the same deletion point that serves
+TTL and count eviction, so an evicted entry unlinks its file. Without that aggregate bound the
+directory was limited only per file (256 MiB) and per entry (1000) — a 250 GiB product — which
+left `RESPONSE_TTL_MS` as the only effective limit and made disk use a function of client
+request rate rather than of anything the process controls.
+
+[Decision Log]
+- 목적과 의도: Bound the durable spill directory in aggregate so demoted continuation state cannot consume the host disk.
+- 기존 구현 및 제약 조건: The resident map has an unconditional byte cap and demotes past it, but the disk it demotes onto had only a per-file ceiling and the shared 1000-entry count cap. Retention itself worked — the hour-long TTL did evict — so the gap was a missing budget, not a leak.
+- 검토한 주요 대안: Lower the per-file ceiling; shorten the TTL; sweep the directory on a timer; add a configurable budget key; carry a running byte counter.
+- 선택한 방식: A constant aggregate ceiling checked at the end of the existing prune, evicting oldest-first, with the total recomputed per prune rather than carried as a counter.
+- 다른 대안 대신 이 방식을 선택한 이유: Per-file or TTL changes alter retention semantics other bounds depend on; a timer adds a second owner for eviction; a config key would surface a knob the sibling bounds (count, TTL, per-file) do not have; and a running counter could silently disable the cap if any of the several insertion paths missed an increment, where a walk over at most 1000 entries cannot drift.
+- 장점, 단점 및 영향: Disk use stops tracking client request rate. Ordinary traffic is unaffected because the count cap binds at a comparable point for median-sized payloads; a workload of unusually large continuations loses its oldest spills earlier than the TTL would, surfacing as the existing `previous_response_not_found` continuation miss.
+
 Response-state loading performs a bounded recovery pass for interrupted snapshot writes. It only
 matches regular files named `responses-state.json.ocx.<pid>.<sequence>.tmp`, waits at least 15
 minutes, and skips the current or any live PID. Eligible files are truncated before unlinking so a

tests/responses-state.test.ts

@@ -44,6 +44,8 @@ import {
   responseContinuationRetainedStoreSnapshot,
   runPendingResponseStatePersistForTests,
   setResponseStateByteCapForTests,
+  setSpilledResponseByteCapForTests,
+  getSpilledResponseBytesForTests,
   setResponseStatePersistAttemptHookForTests,
   getStoredResponseBytesForTests,
 } from "../src/responses/state";
@@ -172,6 +174,7 @@ describe("Responses previous_response_id state", () => {
     setPlatformForTests(null);
     resetHardenedStateForTests();
     setResponseStateByteCapForTests(null);
+    setSpilledResponseByteCapForTests(null);
     clearResponseStateForTests();
     rmSync(home, { recursive: true, force: true });
     if (priorHome === undefined) delete process.env["OPENCODEX_HOME"];
@@ -702,6 +705,21 @@ describe("Responses previous_response_id state", () => {
     }) as { input: unknown[] }).input).toHaveLength(3);
   });
 
+  test("evicts oldest spills once the durable set exceeds the disk cap", () => {
+    setResponseStateByteCapForTests(1_024);
+    setSpilledResponseByteCapForTests(20_000);
+    for (let i = 0; i < 6; i += 1) rememberLarge(`resp_spill_budget_${i}`, "x".repeat(8_000));
+    // Without a disk cap all six stay on disk: the RAM cap only moves bytes out of
+    // memory, it never bounds where they land.
+    expect(getSpilledResponseBytesForTests()).toBeLessThanOrEqual(20_000);
+    expect(spillFileNames(home).length).toBeLessThan(6);
+    // Eviction is oldest-first and must not clear the set it was asked to bound.
+    expect(spillFileNames(home).length).toBeGreaterThanOrEqual(1);
+    expect((expandPreviousResponseInput({
+      previous_response_id: "resp_spill_budget_5", input: "next",
+    }) as { input: unknown[] }).input).toHaveLength(3);
+  });
+
   test("does not swap a resident row to a stub before fsync and no-replace publication succeed", () => {
     const events: string[] = [];
     setSpillIoForTest({ record: event => events.push(event) });
@@ -940,6 +958,63 @@ describe("Responses previous_response_id state", () => {
     expect(spillFileNames(home)).toHaveLength(1);
   });
 
+  test("evicts by createdAt, not by insertion order", () => {
+    const realNow = Date.now;
+    try {
+      setResponseStateByteCapForTests(1_024);
+      // Insert the NEWER entry first so insertion order and createdAt order
+      // disagree. Map order alone would evict the newer one; `states` is not a
+      // reliable age index — writeBoundedSnapshot even serializes it reversed.
+      const base = realNow();
+      Date.now = () => base;
+      const beforeNewer = new Set(spillFileNames(home));
+      rememberLarge("resp_order_newer", "x".repeat(8_000));
+      const newerFile = spillFileNames(home).find(name => !beforeNewer.has(name))!;
+
+      Date.now = () => base - 5 * 60_000;
+      const beforeOlder = new Set(spillFileNames(home));
+      rememberLarge("resp_order_older", "x".repeat(8_000));
+      const olderFile = spillFileNames(home).find(name => !beforeOlder.has(name))!;
+      Date.now = () => base;
+
+      // Each spill payload is ~16.2 KB, so 20_000 leaves room for exactly one.
+      setSpilledResponseByteCapForTests(20_000);
+      rememberLarge("resp_order_trigger", "y");
+
+      const dir = responseSpillDirectory(home);
+      expect(existsSync(join(dir, olderFile))).toBe(false);
+      expect(existsSync(join(dir, newerFile))).toBe(true);
+    } finally {
+      Date.now = realNow;
+    }
+  });
+
+  test("breaks createdAt ties on the response id, not on insertion order", () => {
+    const realNow = Date.now;
+    try {
+      setResponseStateByteCapForTests(1_024);
+      const base = realNow();
+      Date.now = () => base;
+      // Same createdAt, inserted in reverse id order: a stable sort alone would
+      // keep insertion order and evict "_b" first.
+      const before = new Set(spillFileNames(home));
+      rememberLarge("resp_tie_b", "x".repeat(8_000));
+      const bFile = spillFileNames(home).find(name => !before.has(name))!;
+      const beforeA = new Set(spillFileNames(home));
+      rememberLarge("resp_tie_a", "x".repeat(8_000));
+      const aFile = spillFileNames(home).find(name => !beforeA.has(name))!;
+
+      setSpilledResponseByteCapForTests(20_000);
+      rememberLarge("resp_tie_trigger", "y");
+
+      const dir = responseSpillDirectory(home);
+      expect(existsSync(join(dir, aFile))).toBe(false);
+      expect(existsSync(join(dir, bFile))).toBe(true);
+    } finally {
+      Date.now = realNow;
+    }
+  });
+
   test("TTL and count eviction delete dedicated spill files and release stub bytes", () => {
     const realNow = Date.now;
     setResponseStateByteCapForTests(1_024);