Tencent-Hunyuan · leviking98z-rgb · Aug 25, 2026 · Aug 25, 2026 · Aug 25, 2026 · Aug 25, 2026
@@ -11,8 +11,7 @@ UniRL ships two mutually exclusive inference engines (`vllm` and `sglang`) — i
 
 ```bash
 uv venv --python 3.12 --seed .venv && source .venv/bin/activate
-export VLLM_USE_PRECOMPILED=1   # else 30+ min CUDA build
-uv pip install -e ".[vllm,train,infer]"
+uv pip install -e ".[vllm,train,infer]" --prerelease=allow
 ```
 
 ## sglang
@@ -41,7 +40,7 @@ bare venv works for every converter except `datasets/droid100/`, which needs tor
 For development tools (lint and tests):
 
 ```bash
-uv pip install -e ".[vllm,train,infer,eval,dev]"
+uv pip install -e ".[vllm,train,infer,eval,dev]" --prerelease=allow
 # or, for the sglang engine:
 uv pip install -e ".[sglang,train,infer,eval,dev]" --prerelease=allow
 ```

@@ -18,8 +18,7 @@ training environment can be installed with:
 ```bash
 uv venv --python 3.12 --seed .venv
 source .venv/bin/activate
-export VLLM_USE_PRECOMPILED=1
-uv pip install -e ".[vllm,train,infer]"
+uv pip install -e ".[vllm,train,infer]" --prerelease=allow
 ```
 
 Both supported engine extras (`vllm` and `sglang`) install PyAV for raw video

@@ -52,6 +52,7 @@ backend:
   fsdp_cfg:
     _target_: unirl.train.configs.FSDPConfig
     param_dtype: bf16
+    master_dtype: fp32
     cpu_offload: false
     mixed_precision: true
     fsdp_mode: full

@@ -23,11 +23,8 @@
 #     hv15's stage config sets enable_lora/max_lora_rank=64 so the adapter
 #     actually applies. This avoids the CUDA-IPC path's SGLang dependency,
 #     which the vllm-omni-only venv (two-venv image) does not provide.
-#   - algorithm.old_logp_source: replay — the separate-ENGINE anchor (the
-#     vllm-omni engine is a separate process even when colocated; without replay
-#     the cross-process rollout<->replay logp gap dilutes the first-epoch ratio
-#     -> flat reward). Every other vllm-omni v2 engine recipe sets this; the
-#     separate recipe predates the fix and omits it.
+#   - algorithm.old_logp_source: rollout — use the sampler-emitted density as
+#     the frozen policy anchor.
 #
 # Boot note: colocate 1x8 lands all 8 hv15 engine replicas on ONE node — the
 # worst-case parallel-boot memcg burst (each loads ~30GB DiT + Qwen2.5-VL TE +
@@ -78,6 +75,7 @@ backend:
   fsdp_cfg:
     _target_: unirl.train.configs.FSDPConfig
     param_dtype: bf16
+    master_dtype: fp32
     cpu_offload: false
     mixed_precision: true
     fsdp_mode: full
@@ -148,11 +146,8 @@ reward:
 algorithm:
   _target_: unirl.algorithms.flowgrpo.FlowGRPO
   stage_attr: diffusion
-  # Separate-ENGINE anchor: the vllm-omni engine is a separate process even when
-  # colocated, so old_logp must be trainer-side replay (not the engine's rollout
-  # logp). Omitting it bakes the cross-process logp gap into the first-epoch
-  # ratio -> diluted GRPO signal / flat reward. (Matches qwen_image_*_vllmomni_v2.)
-  old_logp_source: replay
+  # Keep the sampler-emitted density as the actual old policy.
+  old_logp_source: rollout
   clip_range: 1.0e-4
   clip_schedule: constant
   conditions_cls:

@@ -221,11 +221,11 @@ if [ "${PROFILE}" = "ar-drpo" ]; then
         exit 2
     fi
 else
-    require_torch_flavor "2.11.0+cu129"
+    require_torch_flavor "2.13.0+cu129"
 fi
 if [ "${PROFILE}" = "sd3-vllm-omni" ] || [ "${PROFILE}" = "qwen-omni" ]; then
-    require_dist_version "vllm" "0.20.0"
-    require_dist_version "vllm-omni" "0.20.0"
+    require_dist_version "vllm" "0.27.0+cu129"
+    require_dist_version "vllm-omni" "0.27.0rc1"
 fi
 
 WANDB_OVERRIDES=(

@@ -73,13 +73,14 @@ sglang = [
   "flash-linear-attention[conv1d]==0.5.1 ; sys_platform == 'linux'",
   "av>=14.2,<19 ; sys_platform == 'linux'",
 ]
-# CUDA 12.9 stack: vllm 0.20.0 PyPI wheels are compiled against CUDA 12.9.
+# CUDA 12.9 stack: use vLLM's release-provided cu129 wheel on the H20 fleet's 535 driver.
 vllm = [
-  "vllm==0.20.0 ; sys_platform == 'linux'",
-  "vllm-omni==0.20.0 ; sys_platform == 'linux'",
-  "torch==2.11.0+cu129 ; sys_platform == 'linux'",
-  "torchvision==0.26.0+cu129 ; sys_platform == 'linux'",
+  "vllm @ https://github.com/vllm-project/vllm/releases/download/v0.27.0/vllm-0.27.0%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl ; sys_platform == 'linux' and platform_machine == 'x86_64'",
+  "vllm-omni==0.27.0rc1 ; sys_platform == 'linux'",
+  "torch==2.13.0+cu129 ; sys_platform == 'linux'",
+  "torchvision==0.28.0+cu129 ; sys_platform == 'linux'",
   "torchaudio==2.11.0+cu129 ; sys_platform == 'linux'",
+  "torchcodec==0.16.0+cu129 ; sys_platform == 'linux'",
   "av>=14.2,<19 ; sys_platform == 'linux'",
 ]
 train = [
@@ -165,19 +166,6 @@ environments = [
   "sys_platform == 'linux' and platform_machine == 'x86_64'",
 ]
 
-# vllm's PyPI wheel is manylinux_2_35 and its GitHub +cu129 wheel manylinux_2_31;
-# the fleet pods are glibc 2.28, so vllm must build from sdist on-pod. Its
-# build-requires omits numpy (CMake configure fails), and the build-env torch
-# must match the runtime +cu129 flavor or the compiled kernels link CUDA 13 libs.
-# First build is slow (or export VLLM_USE_PRECOMPILED=1); uv caches it per pod.
-# (Plain-string pin instead of match-runtime: bump together with the torch
-# anchor in the vllm extra above.)
-[tool.uv.extra-build-dependencies]
-vllm = [
-  "numpy",
-  "torch==2.11.0+cu129",
-]
-
 # Non-explicit and ahead of the implicit PyPI default: participates in
 # resolution for ALL packages, including the engines' transitive torch pins.
 [[tool.uv.index]]

@@ -84,6 +84,10 @@ it is the authoritative bundle / pipeline / stage / conditions contract.
   it's `None`; never build the σ tensor inside `generate`.
 - **CFG empty-negative differs per model** (SD3 `""`, Qwen-Image `" "`) — use the
   model's canonical upstream value or the rollout/replay ratio drifts off 1.0.
+- **HunyuanVideo-1.5 prompt-template whitespace is tokenizer state.** Keep
+  `PROMPT_TEMPLATE_SYSTEM_MESSAGE` byte-identical to upstream because
+  `mllm_crop_start=108` is tied to that exact prefix length; collapsing its
+  indentation drops user-prompt tokens.
 - **Work that needs real storage goes through `types/post_materialize.py`.** A
   bundle or a structural injector (LoRA / NFT / mirror) may run while the module
   is still on the meta device, where writing a tensor is a no-op. Register such

@@ -9,7 +9,8 @@
 
 from unirl.models.types.diffusion import DiffusionStage, DiffusionStep
 from unirl.models.types.replay_result import ReplayResult
-from unirl.sde.kernels import StepStrategy
+from unirl.sde.kernels import GeneratorLike, StepStrategy
+from unirl.sde.noise import make_denoise_step_generators
 from unirl.types.sampling import DiffusionSamplingParams, compute_trajectory_positions
 from unirl.types.segments.latent import LatentSegment, make_video_segment
 from unirl.utils.dtypes import parse_torch_dtype
@@ -78,7 +79,8 @@ def predict_noise(
             timestep = sigma.expand(batch_size)
         else:
             timestep = sigma
-        timestep = timestep.to(device=device, dtype=dtype) * self.TIMESTEP_SCALE
+        # Match Diffusers/vLLM: scale fp32 sigma before casting to the transformer dtype.
+        timestep = (timestep.to(device=device, dtype=torch.float32) * self.TIMESTEP_SCALE).to(dtype=dtype)
 
         latent_model_input = torch.cat([sample_cast, cond_latents, cond_mask], dim=1)
 
@@ -139,6 +141,7 @@ def forward(
         sigma: torch.Tensor,
         sigma_next: torch.Tensor,
         prev_sample: Optional[torch.Tensor] = None,
+        generator: GeneratorLike = None,
         sigma_max: float = 0.99,
         eta: float = 1.0,
         step_index: int = 0,
@@ -151,6 +154,7 @@ def forward(
             sigma_next=sigma_next,
             eta=eta,
             prev_sample=prev_sample,
+            generator=generator,
             sigma_max=sigma_max,
             step_index=step_index,
         )
@@ -166,6 +170,7 @@ def step(
         sigma_next: torch.Tensor,
         guidance_scale: float,
         prev_sample: Optional[torch.Tensor] = None,
+        generator: GeneratorLike = None,
         sigma_max: float = 0.99,
         eta: float = 1.0,
         step_index: int = 0,
@@ -189,6 +194,7 @@ def step(
             sigma=sigma,
             sigma_next=sigma_next,
             prev_sample=prev_sample,
+            generator=generator,
             sigma_max=sigma_max,
             eta=eta,
             step_index=step_index,
@@ -205,6 +211,7 @@ def step_with_logp(
         sigma_next: torch.Tensor,
         guidance_scale: float,
         prev_sample: Optional[torch.Tensor] = None,
+        generator: GeneratorLike = None,
         sigma_max: float = 0.99,
         eta: float = 1.0,
         step_index: int = 0,
@@ -221,6 +228,7 @@ def step_with_logp(
             sigma_next=sigma_next,
             guidance_scale=guidance_scale,
             prev_sample=prev_sample,
+            generator=generator,
             sigma_max=sigma_max,
             eta=eta,
             step_index=step_index,
@@ -305,6 +313,8 @@ def diffuse(
         schedule: torch.Tensor,
         params: DiffusionSamplingParams,
         initial_latents: Optional[torch.Tensor] = None,
+        denoise_seed_keys: Optional[List[str]] = None,
+        denoise_base_seed: int = 0,
     ) -> LatentSegment:
         """Run full HunyuanVideo-1.5 T2V sampling."""
         from unirl.sde.noise import generate_latents
@@ -379,6 +389,15 @@ def diffuse(
             sigma = schedule[i].to(device)
             sigma_next = schedule[i + 1].to(device)
             step_eta = float(params.eta) if i in sde_set else 0.0
+            step_generators = (
+                make_denoise_step_generators(
+                    base_seed=int(denoise_base_seed),
+                    step_index=i,
+                    sample_ids=denoise_seed_keys,
+                )
+                if step_eta > 0.0 and denoise_seed_keys is not None
+                else None
+            )
 
             with torch.no_grad(), autocast_ctx:
                 new_latents, log_prob, _ = self.step.step_with_logp(
@@ -390,6 +409,7 @@ def diffuse(
                     sigma_next=sigma_next,
                     guidance_scale=float(params.guidance_scale),
                     eta=step_eta,
+                    generator=step_generators,
                     sigma_max=sigma_max,
                     step_index=i,
                     **step_kwargs,

@@ -191,7 +191,14 @@ def generate(self, sample: Sample) -> Sample:
 
         initial_latents = NoiseRecipe.from_sample(sample).resolve()
 
-        latent_seg = self.diffusion.diffuse(hv_conds, schedule=schedule, params=params, initial_latents=initial_latents)
+        latent_seg = self.diffusion.diffuse(
+            hv_conds,
+            schedule=schedule,
+            params=params,
+            initial_latents=initial_latents,
+            denoise_seed_keys=list(frontier.sample_ids) if initial_latents is not None else None,
+            denoise_base_seed=int(params.seed) if params.seed is not None else 0,
+        )
         videos = self.vae_decode.decode(latent_seg)
 
         filled = frontier.fill(segment=latent_seg, primitives={"video": videos}, conditions=hv_conds.to_dict())

@@ -12,14 +12,14 @@
 
 from .bundle import HunyuanVideo15Bundle
 
-PROMPT_TEMPLATE_SYSTEM_MESSAGE = (
-    "You are a helpful assistant. Describe the video by detailing the following aspects: "
-    "1. The main content and theme of the video. "
-    "2. The color, shape, size, texture, quantity, text, and spatial relationships of the objects. "
-    "3. Actions, events, behaviors temporal relationships, physical movement changes of the objects. "
-    "4. background environment, light, style and atmosphere. "
-    "5. camera angles, movements, and transitions used in the video."
-)
+# fmt: off
+PROMPT_TEMPLATE_SYSTEM_MESSAGE = "You are a helpful assistant. Describe the video by detailing the following aspects: \
+        1. The main content and theme of the video. \
+        2. The color, shape, size, texture, quantity, text, and spatial relationships of the objects. \
+        3. Actions, events, behaviors temporal relationships, physical movement changes of the objects. \
+        4. background environment, light, style and atmosphere. \
+        5. camera angles, movements, and transitions used in the video."
+# fmt: on
 
 _GLYPH_PATTERN = re.compile(r"\"(.*?)\"|“(.*?)”")
 

@@ -35,6 +35,9 @@ def build_prompts(self, sample: Sample) -> List[Any]:
     def build_sampling(self, sample: Sample) -> List[StageSampling]:
         sampling = super().build_sampling(sample)
         sampling[0].kwargs["num_frames"] = _num_frames(sample)
+        frontier = sample.frontier_gen_part(DiffusionSamplingParams)
+        extra_args = sampling[0].kwargs.setdefault("extra_args", {})
+        extra_args["denoise_seeds"] = [str(sample_id) for sample_id in frontier.sample_ids]
         return sampling
 
 
@@ -113,6 +116,12 @@ class Hv15T2vAdapter(ModelAdapter):
     stage_yaml = "hunyuan_video15_t2v_rl.yaml"
     needs_driver_tokenizer = False
 
+    def boot_kwargs(self) -> Dict[str, Any]:
+        """Pin the vLLM diffusion kernel to the trainer's SDPA path."""
+        kwargs = super().boot_kwargs()
+        kwargs["diffusion_attention_backend"] = "TORCH_SDPA"
+        return kwargs
+
     def __init__(self, config: Any, model_config: Any, *, strategy: Any = None, tokenize_fn: Any = None) -> None:
         super().__init__(config, model_config, strategy=strategy, tokenize_fn=tokenize_fn)
         self.input_adapter = Hv15InputAdapter(self.modality)

@@ -113,6 +113,9 @@ def __init__(
     @classmethod
     def boot(cls, intent: Dict[str, Any]) -> "VLLMOmniBackend":
         """Spell the intent into ``Omni`` ctor kwargs and spawn."""
+        if attention_backend := intent.get("diffusion_attention_backend"):
+            os.environ["DIFFUSION_ATTENTION_BACKEND"] = str(attention_backend)
+
         from unirl.rollout.engine.vllm_omni.patches import install as install_patches
 
         install_patches()

@@ -42,6 +42,8 @@ All in `runtime.py` unless noted.
 | `patch_dit_lora_loader` / `patch_ar_lora_loader` | Stock `DiffusionLoRAManager._load_adapter` loads only from a file path; RL pushes freshly-trained adapter tensors without a disk round-trip (`OmniTensorLoRARequest`). Lifted verbatim from verl-omni | vllm-omni's LoRA managers accept tensor-bag requests natively |
 | `patch_dit_hi3_lora_weights` | Resolves HI3 DiT `transformer.layers.*` wrappers against PEFT `model.layers.*` keys and converts GQA-interleaved fused-QKV LoRA-B rows to vLLM's packed `[q, k, v]` slices | [vllm-omni #6411](https://github.com/vllm-project/vllm-omni/issues/6411) is fixed and the pinned release includes it |
 | `patch_fp32_skip` | Punica kernels hard-assert dtype; HI3's MoE router gate is fp32, so non-fp16/bf16 layers must be skipped for LoRA wrapping | vllm's `from_layer` skips unsupported dtypes itself |
+| `patch_hv15_packed_lora_mapping` | HV1.5 exposes `packed_modules_mapping`, while the diffusion LoRA manager only reads a model-level `stacked_params_mapping`. Consequently all six logical Q/K/V targets per main block are loaded but neither packed projection is wrapped or activated. Translate the model's packed relationship into the attribute consumed by the manager | vllm-omni's LoRA manager reads HV1.5's `packed_modules_mapping` directly |
+| `patch_hv15_refiner_torch_linear_lora` | Pinned vllm-omni's diffusion LoRA manager replaces only vLLM linear classes. HV1.5 deliberately keeps the two token-refiner blocks as Diffusers `nn.Linear` layers, so their 12 configured LoRA targets are loaded and checksum-verified but never applied to the rollout policy. Wrap just those ordinary linears with the same BF16 base-plus-adapter arithmetic and fail if any configured refiner target is missing | vllm-omni applies diffusion LoRA to HV1.5's ordinary token-refiner linears and verifies active-layer coverage |
 | `patch_lora_request_passthrough` | `Omni.generate` never forwards `lora_request`, needed by the HI3 AR-prelude stage. Verified still absent at upstream main (~v0.22.0rc1); `AsyncOmniEngine.add_request` has accepted the kwarg all along, so a small upstream PR forwarding it would retire this | vllm-omni upstreams the kwarg (then the `ar_lora_passthrough` gate drops too) |
 | `patch_per_request_ar_seed` | One `SamplingParams` is shared across requests, so a GRPO group's N requests collapse to identical tokens | vllm-omni stops sharing one `SamplingParams` |
 | `patch_qwen3_omni_thinker_lora` | Backport of vllm-omni #3915: expose the Thinker LoRA interface, select `thinker_config` during model init, accept the current M-RoPE signature | pin vllm-omni ≥ 0.22 |
@@ -82,3 +84,6 @@ known-invalid adapter.
 - **Every patch needs a DELETE-WHEN row.** Without one it is permanent by default.
 - **`patch_hi3_flow_alignment` self-skips on newer pins** — it is dead code, not a
   live patch, once the pin moves past v0.20.0.
+- **The pinned diffusion selector ignores `engine_args.attention_backend`.** A
+  model adapter that needs a fixed kernel must export
+  `DIFFUSION_ATTENTION_BACKEND` through its boot intent before `Omni()` spawns.