fix: support multi-process DDP online rollout

[0z5a]

Description

Enable online_grpo and online_dpo with multi-process DDP without NCCL deadlocks from rank-divergent rollout lengths.

Rollout inference uses the rank-local replicated DDP.module; every training forward/backward/update remains on the DDP wrapper. The change also adds an executor-owned inference boundary, keeps FSDP and torch.compile online rollout fail-fast, fixes rank-local state-dict broadcast allocation, and parameterizes the integration workload through ASTRAI_DDP_TEST_WORLD_SIZE.

No throughput uplift is claimed. This is a correctness, synchronization, and deadlock-freedom change.

NVIDIA L20 validation

Environment: L20 SM89, driver 570.86.10, CUDA 12.8, PyTorch 2.11.0+cu128, NCCL.

• 2-GPU focused suite: 20 passed in 24.42s.
• 6-rank divergent rollouts plus online-GRPO: 2 passed in 13.69s; 7 optimizer steps/rank and checkpoint epoch_0_step_7.
• Five fresh-process 6-rank GRPO replays: 11.85, 11.98, 12.30, 12.18, 12.00s (CV 1.47%).
• 6-rank non-extension regression: 577 passed in 61.54s.
• Integrated PR fix: support multi-process DDP online rollout #55 + fix: keep async rollouts version-consistent #59 soak on 3 L20s: 100,000/100,000 serialized DDP updates in 2,486.142s, 10,001 cross-rank parameter-digest checks, 0 mismatches.
• Long-soak rank-max step latency: median 25.935ms, p95 35.636ms, p99 39.387ms.
• CUDA memory remained 16.453 MiB allocated / 22.0 MiB reserved, with 0 MiB drift.

The integrated soak is recorded against validation revision fe81f17772db7acb8d7575b3f617a454a83ca58d. It combines this DDP path with #59's rollout-version fencing; it is integration evidence, not a performance comparison.

GPU 7 had an active unrelated workload and was not touched. A 7-rank attempt started successfully but was terminated when other users' jobs resumed, so only the completed 3-rank soak is claimed.

Local and InfraSWE validation

• Full local suite: 646 passed, 176 skipped.
• Ruff format/import checks: passed.
• InfraSWE latest a955e00cc3ac79b261d515fb6dd393ba5fd306dd: 283 passed, comparison cell valid.
• Diagnostic ProjectFit 95.00/100, BenchmarkTrust 97.40/100; official score remains unresolved because the draft is unsealed and lacks hidden/system-trace evidence.

Raw environment, replay statistics, limitations, and the long-soak summary are checked in under docs/benchmarks/ and benchmarks/results/.

Compatibility boundaries

• Multi-process online rollout requires parallel_mode='ddp'.
• Distributed FSDP rollout remains unsupported because parameters are sharded.
• Online rollout with torch.compile is rejected before scheduler construction.
• The isolated cu128 runtime lacked nvcc; the L20 regression excludes tests/extension.

Checklist

• Style and import checks pass
• Added regression and integration tests
• Added checked-in benchmark evidence
• No downstream dependency changes