Return best iterate from AMICATorchNG.fit (#51)

[neuromechanist]

Closes #51.

Problem

Multi-model NG log-likelihood was ~0.02 lower and ~13x more variable than Fortran (n_models=2, 100 iters, real sample EEG), even though the M-step is bit-exact vs Fortran (#27).

Root cause: return-last, not a bad basin

fit() returned the last EM iterate under the deliberately non-monotone lrate schedule (both NG and Fortran anneal the rate only after an LL decrease, amica15.f90:1038-1058). An NG-only 20-seed sweep localized the variance to a late overshoot: the sole variance-driving seed climbed to LL -3.3573 (in the pack) by iter 97, then crashed to -3.5452 in its final ~2 iterations after Newton went non-positive-definite and the lrate ramp re-inflated the step. 9/20 "good" seeds also ended below their own peak.

Fix

AMICATorchNG.fit returns the highest-LL iterate:

• keep_best: bool = True restores the best iterate when the run ends more than _KEEP_BEST_TOL (1e-9) below its peak. A monotone single-model run has best == last, so no restore fires and issue Verify NG init-basin vs Newton bit-parity via Fortran load_* init-matching #24 parity stays byte-for-byte identical (verified: max param diff 0.0 with keep_best on vs off). Inactive under do_reject (the good-sample set, hence the LL normalization, changes across iterations, making per-iteration LLs incomparable).
• final_ll_ reports the returned iterate's LL; ll_history stays the true trajectory. The AMICA wrapper, validate_implementations.py, and both ensemble scripts read final_ll_.
• state_dict format bumped 2 → 3 (adds keep_best, final_ll).

Results (ensemble_ll.py, N=20, real EEG + amica15mac, matched schedule)

max_iter=100	mean LL	sd	mean gap	sd ratio
Fortran	-3.3541	0.0031	--	--
NG return-last	-3.3738	0.0399	-0.0197	12.7x
NG keep_best	-3.3634	0.0064	-0.0093	2.0x

keep_best cuts the LL variance from 12.7x → 2.0x Fortran's sd (the headline "~13x more variable" defect).

The residual mean gap is convergence speed, not a worse optimum

NG keep_best mean LL vs iteration budget (8 seeds): -3.3639 @100, -3.3541 (== Fortran) @200, -3.3523 @300. NG needs ~2x iterations for the same LL but converges to the identical optimum (the M-step is bit-exact vs Fortran). Not a correctness issue.

Tests

• test_keep_best_single_model_is_bit_exact (issue Verify NG init-basin vs Newton bit-parity via Fortran load_* init-matching #24 guard: identical params on vs off)
• test_keep_best_snapshot_restore_roundtrip (snapshot clones, restore reverts)
• test_keep_best_returns_within_tol_of_peak (contract: returned LL within tol of peak, params consistent)
• Full torch suite: 69 passed, 5 skipped. Ruff clean; ty no new diagnostics.

Real sample data + Fortran binary only (NO MOCK). ADR .context/decisions/0003-best-iterate-safeguard.md; analysis + reproduction in .context/issue-51/.

[neuromechanist]

Review response (4 Sonnet reviewers: code, silent-failure, tests, comments)

code-reviewer: no ≥80-confidence issues (ran the new tests + the #24 parity guard, confirmed invariants, 0 new ty diagnostics). Its two sub-threshold notes are addressed below.

Fixed (commit 4e063cf)

• final_ll_ stale on a degenerate stop (silent-failure, critical — backend part). It was left at the last finite ll_history value while the model held diverged parameters. Now set to NaN when stop_reason is degenerate.
• n_kurt_done not rolled back with pdtype (silent-failure medium + code-reviewer nit). The best-iterate snapshot now captures n_kurt_done, so a restored/saved adaptive-PDF model's switch count stays consistent with its pdtype.
• do_reject silently disables the default-on safeguard (silent-failure medium). fit now emits a one-time logger.warning when keep_best and do_reject.
• Restore-skip comment was wrong (comment-analyzer, critical). A singular_ll stop leaves A/W finite-but-singular, not non-finite; the comment now states the real reason (salvaging would pre-empt AMICA wrapper marks is_fitted_ even on a degenerate (nan_ll/singular_ll) fit #50's degenerate-fit contract; state_dict already refuses on stop_reason).
• _KEEP_BEST_TOL scale reference ambiguous (comment-analyzer). Qualified to Fortran's min_dll (normalized), explicitly not the legacy NumPy min_dll (un-normalized).
• Tests (test-analyzer gaps): added test_keep_best_inactive_under_reject (final_ll_ == ll_history[-1] under rejection); expanded the peak test with the keep_best=False differential and an explicit pytest.skip when the run is monotone so restore-branch coverage is visible rather than vacuous; the wrapper save/load test now checks final_ll_. 72 torch tests pass.

Deferred to #50 (with rationale)

• AMICA.fit() sets is_fitted_=True unconditionally on a degenerate fit and does not expose stop_reason (silent-failure, critical — wrapper part). This is exactly the scope of issue AMICA wrapper marks is_fitted_ even on a degenerate (nan_ll/singular_ll) fit #50 (the next task). Multi-model NG log-likelihood is ~0.02 lower and more variable than Fortran #51 fixes the backend so it forwards NaN correctly; the wrapper's is_fitted_/stop_reason_/degenerate-transform contract is being implemented in AMICA wrapper marks is_fitted_ even on a degenerate (nan_ll/singular_ll) fit #50.

Declined (with rationale)

• INFO → WARNING for the restore log (silent-failure, high). A restore is the routine, correct action of a default-on safeguard and fires on most multi-model fits (never on the monotone single-model default). Promoting it to WARNING would be alarm-fatigue noise on normal runs, and the outcome is fully visible via final_ll_/ll_history. The sibling nan_ll/lrate_floor warnings are genuine abnormal stops — a different category. Kept at INFO.
• Dedicated degenerate-stop test not added. The backend is robust on the clean sample EEG (even lrate=50 does not diverge to nan_ll/singular_ll), so a natural degenerate stop is not deterministically inducible without rank-deficient/synthetic input, which conflicts with the NO-MOCK policy. The degenerate branches (final_ll_=NaN, restore-skip) are simple guards; the degenerate stop_reason path is exercised by test_state_dict_refuses_degenerate_model.