feat(minimax-h3): vLLM-Omni rollout backend for H3 FlowGRPO at 32-GPU scale

[nussejzz]

Summary

vLLM-Omni rollout backend for MiniMax-H3 T2VA flow-GRPO, plus the 32-GPU
deployment it runs on. H3 is a 33B dense omni-modal transformer that denoises
video and stereo audio in one packed sequence, so it does not fit the existing
single-latent rollout path.

What this adds:

• Rollout adapter and RL pipeline for H3 under vLLM-Omni: grouped four-GPU
  replicas (DiT TP2 x Ulysses2, text-encoder TP4), sparse FP32 trajectory
  capture, and behaviour-policy log-probs emitted per SDE transition.
• Staged LoRA publication. LocalLoraWeightSync splits into
  extract() / push() so the adapter is extracted while the trainer is
  resident and pushed after trainer offload and rollout wake-up, which is what
  makes three roles time-share the same GPUs.
• LoRA transport verification. add_lora() only registers CPU-side logical
  weights, so a name or shape mismatch against the fused vllm DiT is silent:
  the engine keeps serving base weights and the only symptom is a flat reward
  curve. The Diffusers-to-serving translation now lives in one place
  (renames, the GEGLU half-swap Diffusers stores as [up, gate], synthesized
  rope.inv_freq), every expected block/slice must carry both A and B, and
  per-layer checksums are compared against the awake engine.
• Row-level reward dispatch and a resident reward service at DP32.
• Numerical fixes. Global CUDA autocast ran the fp32-pinned projections,
  timestep MLP and rope in bf16 despite their storage dtype; H3 conditions on
  t = 1 - sigma, where bf16 spacing near 1.0 is ~0.008, so this was real
  precision loss at the low-noise end.
• Rollout contract fixes found at 32-GPU scale: variable-length prompt
  embeddings are padded with a mask and trimmed before the packed forward
  (concatenating them directly raises on any batch whose prompts tokenize to
  different lengths, hit at 32 prompts spanning 9-58 tokens); SDE exploration
  keys come from the logical sample id rather than the engine request UUID, so a
  retried or resumed sample replays the trajectory it actually generated; sparse
  replay always stores the terminal latent for decode.
• Continuation guarantees. Resuming advanced the data source twice when both
  a checkpoint and data_source.start_batch were set; a checkpoint save that
  wrote no model state returned successfully.
• Dependency floor moved to vLLM 0.27 / PyTorch 2.13 cu130 / Transformers 5.10+.

Related Issue

Depends on vllm-project/vllm-omni#6351 (open). Includes the H3 trainside commit
series from #347 rebased onto current main, so #347 and this PR must be
coordinated rather than merged independently.

Test Plan

Static, on this branch rebased onto main:

• pytest tests/ — 16 passed (LoRA layout mapping and coverage, video PickScore
  frame sampling, T2AV composite failure propagation, stable SDE sample key).
• ruff check + ruff format --check on all 42 changed Python files — clean.
• python lint/check_docstring_lines.py (3284 docstrings),
  lint/check_recipe_targets.py (2517 _target_ paths resolve),
  lint/check_core_dependencies.py, lint/check_experimental_boundaries.py —
  all pass.
• Hydra composition:
  python -m unirl.train_diffusion --config-name diffusion/minimax_h3/minimax_h3_t2va_vllmomni_32c_quality100_tp2_up2 --cfg job.

On hardware — 4 nodes x 8 x H20 96GB, real MiniMax-H3 33B, LoRA rank 64,
256x448x107, 24 transitions:

• Two-rollout smoke through the full lifecycle (LoRA sync -> rollout wake ->
  reward -> trainer replay/step -> sleep), then an 11-rollout run at 32 prompts
  x 8 samples with CLAP + ImageBind. Rollout/replay parity held at
  |dlogp| ~1e-5 against a 1e-3 gate; PPO ratio 1.0000; LoRA verification
  reported 350 modules loaded per replica; no OOM.
• Rollout topology comparison at K8 (8 prompts x 8 samples): HSDP4+UP4 median
  generate 112.697s / 66,876 MiB peak; TP2 x UP2 91.087s / 77,436 MiB, parity
  drift 3.73e-5 and 3.94e-5. A TP4 arm raced a stale GPU keep-alive and OOM'd,
  so that number was discarded and TP2 x UP2 is the fastest verified arm, not
  a proven optimum.

Not run:

• Regression of the existing BAGEL / HI3 / Qwen3-Omni vLLM-Omni recipes under
  the vLLM 0.27 / Torch 2.13 dependency upgrade. This is the main outstanding
  risk and needs an owner with those recipes' checkpoints.
• A clean CU13 environment rebuild from scratch.

Compatibility / Risk

• Not mergeable as-is. pyproject.toml pins
  vllm-omni @ git+https://github.com/nussejzz/vllm-omni.git@a4759624, a fork
  commit, because [Core][Diffusion] Propagate RPC policy and support packed LoRA targets vllm-project/vllm-omni#6351 is still open. Replace with the
  first upstream release containing #6351 before merge.
• Global dependency floor moves to vLLM 0.27 / Torch 2.13 cu130 /
  Transformers 5.10+, which affects every vLLM-Omni recipe, not just H3. The
  SGLang environment stays independent.
• LocalLoraWeightSync.sync() is preserved as extract() + push(), so
  existing callers are unaffected.
• H3 geometry rejects anything off the released 768-short-edge distribution
  unless a recipe opts in with sampler_kwargs.allow_nonstandard_canvas: true.
• Adds a top-level tests/ directory, which this repo does not currently have
  and which no CI workflow runs. Happy to relocate or drop it if that is not
  the direction you want.

Reviewer Notes

Reward convergence is not solved and this PR does not claim it. The
topology, lifecycle, LoRA sync and parity gates pass, and the throughput
numbers above are reproducible, but no H3 recipe here produces a sustained rise
in held-out visual reward. A Wan2.1 positive control on the same trainer,
PickScore and GRPO path does rise (0.6985 -> 0.7201 over 20 rollouts), which
is what localizes the open problem to H3/reward rather than to the shared
training stack. Treat this as backend and deployment support, not as a tuned
recipe.

One merge resolution needs your judgement. The rebase onto current main hit a
semantic conflict in _validate_prompt_tree_dp_geometry: main now validates
reward DP against batch_size (prompt trees), while this branch validates
against batch_size * samples_per_prompt (independently scored rows). I kept
the row-based rule, because the H3 deployment runs 8 prompts x 16 samples over
reward DP32 and ran 11 rollouts that way, and because row-based is strictly more
permissive — anything the tree rule accepts, it also accepts, so no existing
recipe can start failing. But RewardService.score_and_attach is
Dispatch.DP_SCATTER, so the tree rule may be the correct invariant for the
general path and I did not want to decide that unilaterally.

The flow-GRPO algorithm knobs and the experiment recipes are deliberately not
here; they are stacked on this branch in a follow-up so this PR stays reviewable
as backend support. A DiffusionNFT counterpart is planned separately.

AI assistance was used for this change.

Checklist

• I reviewed the changed code and removed unrelated/generated artifacts.
• I updated tests, docs, and configs where needed, or explained why not.