Skip to content

Commit 3ab718b

Browse files
committed
Close TODO bug #1: Comparator removes the SafeVerify memory blowup
The DAG-shaped text export + kernel replay have no importModules in the trusted process and no un-memoized rebuildExpr, retiring both unbounded-memory sources. mem_limit drops 50g -> 16g pending the §7 peak-RSS measurement.
1 parent 5f947fe commit 3ab718b

1 file changed

Lines changed: 17 additions & 14 deletions

File tree

TODO.md

Lines changed: 17 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,26 @@
22

33
## Known bugs
44

5-
### 1. SafeVerify peak memory is effectively unbounded on legitimate proofs (un-memoized `rebuildExpr`)
5+
### 1. SafeVerify peak memory is effectively unbounded on legitimate proofs (un-memoized `rebuildExpr`) — RESOLVED by the Comparator migration
66

7-
**Severity:** medium — causes deterministic scorer OOM kills (infra errors / lost samples),
8-
not mis-scoring. Already documented in code; tracked here for visibility.
7+
**Status:** resolved on the `comparator` branch (see `comparator-migration-plan.md`).
8+
The verifier no longer materializes the full Mathlib environment or deep-copies
9+
proof terms: Comparator consumes lean4export's *text* export, which serializes
10+
terms as a shared DAG (every subterm emitted once, by index) and replays that
11+
through its kernel. There is no `importModules` in the trusted process and no
12+
un-memoized `rebuildExpr`, so the two failure sources below are gone. The
13+
comparator service starts at `mem_limit: 16g` (down from `50g`), to be confirmed
14+
by the §7 peak-RSS measurement — in particular re-running the `(a+b+c)^16` ring
15+
case and the three formerly resource-bound gold proofs
16+
(`RESOURCE_BOUND_STEMS` in `tests/test_gold_proofs.py`).
917

10-
**Detail** (see `apn/task.py:96-121` and `apn/checker.py:36-44`):
11-
- `safe_verify` has a large fixed footprint (~27 GiB peak RSS), attributed almost
18+
**Original detail** (SafeVerify, now retired):
19+
- `safe_verify` had a large fixed footprint (~27 GiB peak RSS), attributed almost
1220
entirely to four `importModules` calls (two in the import-superset check, one per
1321
replayed file), each materializing the full Mathlib environment and never freeing it.
14-
- On top of that, proof *content* is unbounded: `rebuildExpr` deep-copies proof terms
22+
- On top of that, proof *content* was unbounded: `rebuildExpr` deep-copied proof terms
1523
**without memoization**, expanding pointer-shared DAGs (which tactics like `ring`
16-
produce routinely) into trees. Measured: `(a+b+c)^16 = (c+b+a)^16 := by ring` compiles
24+
produce routinely) into trees. Measured: `(a+b+c)^16 = (c+b+a)^16 := by ring` compiled
1725
agent-side in 3.5s at 6.4 GiB but blew past a 34 GiB limit in safe_verify before being
18-
OOM-killed. Real submissions have reached ~43 GiB in production. `mem_limit` is set to
19-
`50g` to cover the worst observation, but no limit can make scorer OOMs impossible.
20-
- Agent-side compile success does **not** bound the scorer's cost.
21-
22-
**Possible fixes** (in vendored `safeverify`):
23-
- Memoize `rebuildExpr` so shared sub-terms are copied once.
24-
- Skip the redundant `importModules` in the import-superset check.
26+
OOM-killed. Real submissions reached ~43 GiB in production.
27+
- Agent-side compile success did **not** bound the scorer's cost.

0 commit comments

Comments
 (0)