|
2 | 2 |
|
3 | 3 | ## Known bugs |
4 | 4 |
|
5 | | -### 1. SafeVerify peak memory is effectively unbounded on legitimate proofs (un-memoized `rebuildExpr`) |
| 5 | +### 1. SafeVerify peak memory is effectively unbounded on legitimate proofs (un-memoized `rebuildExpr`) — RESOLVED by the Comparator migration |
6 | 6 |
|
7 | | -**Severity:** medium — causes deterministic scorer OOM kills (infra errors / lost samples), |
8 | | -not mis-scoring. Already documented in code; tracked here for visibility. |
| 7 | +**Status:** resolved on the `comparator` branch (see `comparator-migration-plan.md`). |
| 8 | +The verifier no longer materializes the full Mathlib environment or deep-copies |
| 9 | +proof terms: Comparator consumes lean4export's *text* export, which serializes |
| 10 | +terms as a shared DAG (every subterm emitted once, by index) and replays that |
| 11 | +through its kernel. There is no `importModules` in the trusted process and no |
| 12 | +un-memoized `rebuildExpr`, so the two failure sources below are gone. The |
| 13 | +comparator service starts at `mem_limit: 16g` (down from `50g`), to be confirmed |
| 14 | +by the §7 peak-RSS measurement — in particular re-running the `(a+b+c)^16` ring |
| 15 | +case and the three formerly resource-bound gold proofs |
| 16 | +(`RESOURCE_BOUND_STEMS` in `tests/test_gold_proofs.py`). |
9 | 17 |
|
10 | | -**Detail** (see `apn/task.py:96-121` and `apn/checker.py:36-44`): |
11 | | -- `safe_verify` has a large fixed footprint (~27 GiB peak RSS), attributed almost |
| 18 | +**Original detail** (SafeVerify, now retired): |
| 19 | +- `safe_verify` had a large fixed footprint (~27 GiB peak RSS), attributed almost |
12 | 20 | entirely to four `importModules` calls (two in the import-superset check, one per |
13 | 21 | replayed file), each materializing the full Mathlib environment and never freeing it. |
14 | | -- On top of that, proof *content* is unbounded: `rebuildExpr` deep-copies proof terms |
| 22 | +- On top of that, proof *content* was unbounded: `rebuildExpr` deep-copied proof terms |
15 | 23 | **without memoization**, expanding pointer-shared DAGs (which tactics like `ring` |
16 | | - produce routinely) into trees. Measured: `(a+b+c)^16 = (c+b+a)^16 := by ring` compiles |
| 24 | + produce routinely) into trees. Measured: `(a+b+c)^16 = (c+b+a)^16 := by ring` compiled |
17 | 25 | agent-side in 3.5s at 6.4 GiB but blew past a 34 GiB limit in safe_verify before being |
18 | | - OOM-killed. Real submissions have reached ~43 GiB in production. `mem_limit` is set to |
19 | | - `50g` to cover the worst observation, but no limit can make scorer OOMs impossible. |
20 | | -- Agent-side compile success does **not** bound the scorer's cost. |
21 | | - |
22 | | -**Possible fixes** (in vendored `safeverify`): |
23 | | -- Memoize `rebuildExpr` so shared sub-terms are copied once. |
24 | | -- Skip the redundant `importModules` in the import-superset check. |
| 26 | + OOM-killed. Real submissions reached ~43 GiB in production. |
| 27 | +- Agent-side compile success did **not** bound the scorer's cost. |
0 commit comments