Skip to content

M7_NOTES.md

Latest commit

 

History

History
134 lines (104 loc) · 4.74 KB

M7_NOTES.md

File metadata and controls

134 lines (104 loc) · 4.74 KB
Raw

M7 — Checkpoint with incremental hits persistence

Problem

Pre-M7, the program saved only a textual checkpoint with the last completed (C, z) pair. Hits accumulated in RAM as a std::vector and were only flushed to stdout at the very end. Any interruption mid-run (Ctrl-C, kill, system crash, power loss) would lose all hits. Resume could continue from the checkpoint, but the new run would start all_hits empty — losing forever the hits found before the interruption.

For overnight production runs of 8-20 hours, this was unacceptable risk.

Solution

Three changes:

  1. Hits file (<checkpoint>.hits) — binary, append-only.

    • 28-byte records: A, x, B, y, C, z, gcd as uint32_t.
    • Appended after each (C, z) cycle completes.
    • fflush + fsync after each write — survives OS crash.

  2. Resume reads hits file before starting main loop.

    • Reconstructs all_hits from disk on startup.
    • Reports [resume] loaded N previously-saved hits.

  3. SIGINT handler (also catches SIGTERM).

    • Sets a global flag instead of immediately exiting.
    • Main loop checks flag after each (C, z) — exits cleanly with state saved.
    • Checkpoint and hits file are PRESERVED (not deleted) on interrupt.
    • Files only deleted if run completes successfully end-to-end.

Usage

The wrapper scripts/long_run.sh enables checkpoint by default. Manual:

# Start
./beal_bigint 100000 3 15 --raw --skip-gmp-verify \
    --checkpoint results/run.cp \
    --result-file results/run.txt \
    --ledger results/coverage_ledger.csv

# Stop cleanly mid-run (waits for current C,z, then exits):
pkill -INT beal_bigint

# Resume (same flags + --resume):
./beal_bigint 100000 3 15 --raw --skip-gmp-verify \
    --checkpoint results/run.cp \
    --result-file results/run.txt \
    --ledger results/coverage_ledger.csv \
    --resume

Caveats

  • Hits file is binary. Tools to inspect:

    # Number of hits stored
echo "$(stat -c%s results/run.cp.hits) / 28" | bc

# Read in Python:
import struct
with open("results/run.cp.hits", "rb") as f:
    while chunk := f.read(28):
        A, x, B, y, C, z, g = struct.unpack("7I", chunk)
        print(f"{A}^{x} + {B}^{y} = {C}^{z}  gcd={g}")

  • Session counters reset on resume: total_candidates, total_coprime, and total_fps reflect only the SESSION (after resume), not cumulative across sessions. The hit list itself is correctly cumulative.

  • Don't change command flags between original run and --resume. Changing bound, expmin, expmax, or --c-min/--c-max will produce inconsistent results.

  • Hits file grows ~28 bytes per hit. A run with 100k hits → ~2.8 MB. Negligible disk usage.

File lifecycle

Event checkpoint hits file
Run starts (new) created (empty) created (empty)
Each (C, z) completes overwritten appended
SIGINT received preserved preserved
Run completes successfully deleted deleted

Why not save after every hit?

Per-hit fsync would devastate throughput. Per-(C,z) fsync is the natural checkpoint boundary — at minimum thousands of (C,z) pairs per overnight run, so granularity is fine.

M7.1 — Auto-resume default

Change

Removed the requirement to pass --resume to continue an interrupted run. If a checkpoint file exists at the given --checkpoint PATH, the program now resumes automatically with a clear notice:

[auto-resume] checkpoint found at (C=314, z=14); continuing.
              Use --restart to discard and start from zero.

Rationale

  • "Resume if checkpoint exists" is the natural expected behavior for any long-running tool with persistent state (git rebase --continue, tmux attach, rsync, etc).
  • Reduces friction: typing --resume after a crash was an extra step.
  • Eliminates the --resume --resume duplication that the wrapper produced when the user passed --resume to a wrapper that already adds it.

New flag: --restart

Explicit opt-in to discard existing checkpoint+hits and start from zero. Required because, with auto-resume default, there's no other safe way to re-run the same command from scratch.

Backwards compatibility

--resume is still accepted as a no-op. Old scripts/notes that pass it continue to work. Will be removed in a future major version.

Lifecycle (updated)

Event checkpoint hits file
Run starts (no existing) created (empty) created (empty)
Run starts, checkpoint exists auto-resumed auto-loaded
Run starts, checkpoint exists, --restart DELETED + created fresh DELETED + created fresh
Each (C, z) completes overwritten appended
SIGINT received preserved preserved
Run completes successfully deleted deleted