Skip to content

fix(hunyuan-video15): align vLLM-Omni rollout policy - #390

Open
leviking98z-rgb wants to merge 20 commits into
Tencent-Hunyuan:mainfrom
leviking98z-rgb:fix/hunyuan-video15-vllm-trainside-alignment
Open

fix(hunyuan-video15): align vLLM-Omni rollout policy#390
leviking98z-rgb wants to merge 20 commits into
Tencent-Hunyuan:mainfrom
leviking98z-rgb:fix/hunyuan-video15-vllm-trainside-alignment

Conversation

@leviking98z-rgb

@leviking98z-rgb leviking98z-rgb commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Align HunyuanVideo-1.5 trainside and vLLM-Omni at the input, stochastic
transition, checkpoint, and online-LoRA boundaries:

  • preserve the exact upstream MLLM system-prompt whitespace because
    mllm_crop_start=108 is tied to that tokenized prefix;
  • keep fp32 LoRA master parameters on both matched recipes while retaining
    BF16 forward parameters;
  • derive SDE noise from the same (seed, step, sample_id) tuple on both
    engines, preserving per-sample independence across vLLM request splitting;
  • form the transformer timestep as fp32 sigma * 1000 before the BF16 cast,
    matching Diffusers/vLLM;
  • select the trainer's SDPA backend through the small boot-intent path used by
    the pinned vLLM-Omni selector;
  • load the token refiner's unfused Q/K/V checkpoint tensors instead of leaving
    them randomly initialized;
  • expose the main blocks' packed QKV LoRA mapping and apply all 12 configured
    token-refiner nn.Linear LoRA targets, failing fast on incomplete coverage;
    and
  • keep algorithm.old_logp_source: rollout, using the sampler-emitted density
    as the frozen policy anchor.

The cleanup deliberately excludes the pin-specific exact-arithmetic
monkey-patches for autocast, Q/K RMSNorm, SDPA mask construction, and RoPE, as
well as the rejected merged-weight and raw-VAE-reward experiments.

Related Issue

Related to #386.

Test Plan

  • python -m py_compile on all changed Python modules — passed.
  • ruff check and ruff format --check on all changed Python modules —
    passed.
  • pre-commit run --files <all changed files> — all applicable hooks passed,
    including YAML, AST, Ruff, formatting, recipe target, docstring,
    experimental-boundary, and dependency-direction checks.
  • Static AST comparison confirms the trainside prompt template is byte-equal
    to both pinned Diffusers and vLLM-Omni (477 characters).
  • YAML assertions confirm both recipes resolve master_dtype: fp32, the
    vLLM-Omni recipe resolves old_logp_source: rollout, and the stage resolves
    attention_backend: TORCH_SDPA.
  • The prompt-only matched run changed first-rollout reward delta from the
    unpatched +0.0370 to -0.0007, confirming that prompt whitespace is an
    initial-conditioning requirement rather than a cosmetic edit.
  • The scheduler contract produced bit-identical emitted transitions and log
    probabilities on both engines while assigning different SDE noise to
    different samples.
  • The timestep contract found the old trainside ordering produced 976
    versus 980 and 684 versus 680 at two steps of the matched schedule;
    fp32 scaling makes all ten transformer timesteps bit-identical.
  • A full-checkpoint audit found 12 unequal refiner Q/K/V parameters before the
    loader fix. After it, all 15.510 GiB compared bit-identically
    (missing=0, unequal=0).
  • Live LoRA coverage increased from 324 to 444 active wrappers. In the
    loader-only versus full-coverage comparison at rollout 75, correlation
    improved from 0.964188 to 0.992005 and MAE from 0.00398356 to
    0.00226971.
  • The accepted full-coverage arm remained aligned past the suspected failure
    point: correlation/MAE were 0.999703 / 0.00046770 through rollout 10 and
    0.991211 / 0.00354971 through rollout 125. At rollout 140 correlation was
    0.990625 with mean bias +0.00309; no ratio or reward-curve collapse
    occurred:

That live curve included the exact-arithmetic probes now excluded from the
final diff. The selected input/weight/LoRA contracts were validated within
that run, but the narrowed final HEAD has not been rerun on GPU; the PR
therefore remains Draft.

Compatibility / Risk

  • The runtime LoRA/checkpoint patches are gated to the pinned
    HunyuanVideo-1.5 vLLM-Omni model. Existing packed checkpoint loading for the
    54 main blocks remains unchanged.
  • The sample-keyed SDE path is opt-in: it is used only when the request carries
    denoise_seeds; existing callers without those keys keep their prior RNG
    behavior.
  • Token-refiner adapters use two ordinary BF16 linear operations instead of a
    Punica wrapper, adding a small refiner-forward cost.
  • The targeted refiner wrapper supports adapter slot 0, matching the recipe's
    single-online-adapter setup, and rejects an unexpected slot explicitly.
  • No checkpoint schema, dataset format, or trajectory wire-format migration is
    introduced.

Reviewer Notes

The exploratory history is intentionally retained. Commit 89e29069 removes
the broad candidate set additively; commit c95c7768 restores only the
validated input/training contracts and small backend selection path. The final
runtime contains exactly three HunyuanVideo-1.5 monkey-patches, all for
checkpoint or LoRA coverage.

Duplicate-work check: #220 covers HunyuanVideo-1.0 conditioning/merged rollout
weights, while #371 covers the SGLang pipeline/SDE path and supports both
sample-keyed noise and the rollout-logp anchor choice. Neither applies all
HunyuanVideo-1.5 vLLM-Omni packed/refiner LoRA targets. AI assistance was used;
the final diff and reported validation results were reviewed before
publication.

Checklist

  • I reviewed the changed code and removed unrelated/generated artifacts.
  • I updated tests, docs, and configs where needed, or explained why not.

EXP

截屏2026-08-29 04 40 36

@github-actions github-actions Bot added the wip Draft / work in progress label Aug 25, 2026
@leviking98z-rgb leviking98z-rgb changed the title fix(hunyuan-video15): align trainside with vLLM-Omni fix(hunyuan-video15): align vLLM-Omni LoRA policy Aug 26, 2026
@leviking98z-rgb
leviking98z-rgb marked this pull request as ready for review August 26, 2026 17:44
@github-actions github-actions Bot added need review Ready and waiting for review and removed wip Draft / work in progress labels Aug 26, 2026
@leviking98z-rgb leviking98z-rgb changed the title fix(hunyuan-video15): align vLLM-Omni LoRA policy fix(hunyuan-video15): align vLLM-Omni rollout policy Aug 26, 2026
@leviking98z-rgb
leviking98z-rgb marked this pull request as draft August 26, 2026 17:55
@github-actions github-actions Bot removed the need review Ready and waiting for review label Aug 26, 2026
@github-actions github-actions Bot added the wip Draft / work in progress label Aug 26, 2026
@leviking98z-rgb
leviking98z-rgb marked this pull request as ready for review August 26, 2026 18:00
@github-actions github-actions Bot added need review Ready and waiting for review and removed wip Draft / work in progress labels Aug 26, 2026
CjhHa1 and others added 2 commits August 28, 2026 23:18
Drop the HV1.5 loader workaround now provided upstream while retaining the two LoRA coverage patches that 0.27 still needs.
Select the official cu129 vLLM wheel and matching PyTorch packages so the 0.27 stack runs on the H20 fleet's current driver.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

need review Ready and waiting for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants