Skip to content

Fail fast when GRPO combines Liger with the MoE auxiliary loss - #7161

Open
YeonwooSung wants to merge 1 commit into
huggingface:mainfrom
YeonwooSung:fix/7009-grpo-liger-moe-aux
Open

Fail fast when GRPO combines Liger with the MoE auxiliary loss#7161
YeonwooSung wants to merge 1 commit into
huggingface:mainfrom
YeonwooSung:fix/7009-grpo-liger-moe-aux

Conversation

@YeonwooSung

@YeonwooSung YeonwooSung commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

GRPOTrainer already sets aux_loss_enabled for Mixture-of-Experts models, but use_liger_kernel=True routes compute_loss through compute_liger_loss, which never sees router logits. The load-balancing term was dropped with no error.

DPO and KTO already raise in this combination. This copies that fail-fast into GRPO (Liger GRPO loss does not support...) and adds test_init_fails_with_moe_aux_loss_and_liger on tiny-Qwen3MoeForCausalLM.

Fixes #7009

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline, Pull Request section?
  • Was this discussed/approved via a GitHub issue? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

AI writing disclosure

We welcome the use of AI tools to help with contributions. For transparency and to help us improve our review process, please indicate the level of AI involvement in this PR.

  • No AI usage: the PR was written entirely by a human.
  • AI-assisted: some parts were suggested or improved by AI, but the PR was written and reviewed by a human.
  • AI-generated: the PR was mostly or fully generated by an AI tool.

Who can review?

Anyone in the community is free to review the PR once the tests have passed.


Note

Low Risk
Init-time validation only; no change to training paths except blocking an already-unsupported silent misconfiguration.

Overview
GRPO now rejects Liger + MoE aux loss at init, matching DPO/KTO behavior. When a MoE model has load-balancing enabled (router_aux_loss_coef non-zero) and use_liger_kernel=True, GRPOTrainer raises a ValueError explaining that the fused Liger path cannot use router logits, with guidance to disable the aux loss or turn off Liger.

Adds test_init_fails_with_moe_aux_loss_and_liger (Liger-gated) using tiny-Qwen3MoeForCausalLM to assert that error on construction.

Reviewed by Cursor Bugbot for commit 7566981. Bugbot is set up for automated code reviews on this repo. Configure here.

Liger fuses the GRPO loss without materializing router logits, so the
MoE load-balancing term was silently dropped. Raise the same error DPO
and KTO already use.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GRPOTrainer silently drops the MoE auxiliary loss when use_liger_kernel=True

1 participant