feat(train): add versioned dynamic sampling refill

[0z5a]

Summary

• add an explicit per-prompt dynamic-sampling state machine with deterministic refill seeds and hard per-group/per-step budgets
• keep provisional groups behavior-policy-version consistent across refills and invalidate/restart when the live version advances
• synchronize accept/refill/failure decisions across distributed ranks to prevent partial batches and collective hangs
• expose opt-in online-GRPO configuration, CLI flags, metrics, docs, tests, and reproducible L20 benchmarks

Dependency

This is stacked on #59 (fix: keep async rollouts version-consistent). Until #59 merges, the GitHub diff against main also contains that prerequisite. The feature-specific commits start at 392d53d3.

Validation

• local full suite: 650 passed, 171 skipped
• L20 targeted tests: 42 passed
• ruff format/import order: pass
• latest InfraSWE 191b909: 283 passed; minimum training fixture suite: pass
• explicit local InfraSWE D1 target resolution: https://github.com/ViperEkura/AstrAI

L20 benchmark

Single L20, 50 measured trials, 8 prompts x 4 generations, half of first attempts forced zero-variance:

Mode	Median	p95	Generated tokens	Accepted groups / 1M tokens
baseline	248.52 ms	256.62 ms	24,657	8,111.29
dynamic sampling	387.20 ms	396.26 ms	36,974	10,818.41

Dynamic sampling improves effective accepted groups per million generated tokens by 33.37%, with an explicit 55.80% median-latency and 49.95% generated-token cost for this adversarial zero-variance workload.

Three L20 NCCL replay, 100 measured steps with rank-skewed decisions and policy-version jitter:

• mixed-version batches: 0
• incomplete batches: 0
• generation-schedule mismatches: 0
• generation/scoring rank-local failure propagation: pass / pass
• rank-0 reserved-memory drift: 0 bytes

InfraSWE scope

The checked-in InfraSWE score is deliberately provisional and non-leaderboard: ProjectFit 88.37, BenchmarkTrust 93.42. Official status remains unresolved because the Draft is D1/unsealed, hidden probes are incomplete, the manifest is unverified, and the official replay minimum is not met.

Limitations

The benchmark uses a tiny real CUDA policy with deterministic synthetic rewards. It validates lifecycle, versioning, refill efficiency, distributed agreement, and failure propagation; it does not claim downstream reward convergence.