Issues

• ChunkedLogProbFunction returns wrong log-probs and non-finite gradients when the support is far below the row maximum

  huggingface/trl#7982 · Filter by author albertvillanova opened on Oct 28, 2026

• [BUG] GRPO/RLOO continuous batching casts the trained weights to bf16 in place

  huggingface/trl#7965 · Filter by author harshaljanjani opened on Oct 28, 2026

• Popular checkpoints are rejected for assistant-only loss: The chat template is not training-compatible

  huggingface/trl#7962 · Filter by author albertvillanova opened on Oct 28, 2026
  

• Tool calling is not supported for some popular model families

  huggingface/trl#7960 · Filter by author albertvillanova opened on Oct 28, 2026
  

• CI fails with vLLM for PEFT with a tied lm_head: ValueError: 'lm_head.weight' was skipped because it is tied to 'model.embed_tokens.weight'

  huggingface/trl#7955 · Filter by author albertvillanova opened on Oct 28, 2026
  

• CI fails with vLLM for GRPO structured outputs: AssertionError: Parameter model.layers.1.self_attn.k_proj.bias has not changed.

  huggingface/trl#7954 · Filter by author albertvillanova opened on Oct 28, 2026
  

• VESPO with vllm_importance_sampling_mode="token_mask" drops the whole sequence for a single out-of-range token

  huggingface/trl#7952 · Filter by author yusufozdil opened on Oct 27, 2026
  1

• The <think>-format examples score with accuracy_reward, which credits a \boxed{} answer inside the reasoning block

  huggingface/trl#7950 · Filter by author jimmy-holdout opened on Oct 27, 2026

• Popular checkpoints are rejected for tool calling: Unrecognized chat template

  huggingface/trl#7943 · Filter by author albertvillanova opened on Oct 27, 2026
  

• GKD: on-policy steps normalize the loss by the dataset's token count, not the generated one

  🐛 bug
  huggingface/trl#7924 · Filter by author qgallouedec opened on Oct 26, 2026

• [Tracking] Train KTO and distillation over the same tiny models as DPO and GRPO

  huggingface/trl#7923 · Filter by author qgallouedec opened on Oct 26, 2026
  2

• MoE aux loss: AttributeError: ... has no attribute 'num_experts' on Granite MoE, granite-4.0-h and PhiMoE

  huggingface/trl#7918 · Filter by author qgallouedec opened on Oct 26, 2026

• GRPO loss_type="cispo": the default epsilon_high clips every importance weight to 0.2

  🐛 bug
  huggingface/trl#7916 · Filter by author qgallouedec opened on Oct 26, 2026

• GRPO with vLLM: repetition_penalty drives the importance sampling ratio to 0 and stalls training

  🐛 bug
  huggingface/trl#7914 · Filter by author qgallouedec opened on Oct 26, 2026

• DPO reuses stale reference scores when a custom collator's sequence limit changes

  huggingface/trl#7897 · Filter by author JohnLyu2 opened on Oct 25, 2026
  2

• RLOO: clipped sequence ratios produce NaN gradients with a finite loss on reused long completions

  huggingface/trl#7896 · Filter by author taking-lying-flat opened on Oct 25, 2026
  1

• parse_response returns tool calls as plain text when prefix is omitted (transformers >= 5.13)

  huggingface/trl#7895 · Filter by author Sonica-B opened on Oct 25, 2026
  1

• DistillationTrainer ignores Inkling μP scaling and effective output vocabulary

  huggingface/trl#7894 · Filter by author taking-lying-flat opened on Oct 25, 2026
  2

• Training with the default bf16=True crashes on pre-Ampere GPUs (T4, V100, RTX 20xx) since v1.15

  huggingface/trl#7889 · Filter by author akanyaani opened on Oct 24, 2026

• GKD/Distillation reject same-tokenizer pairs that only differ in padded vocab_size (Qwen2.5 0.5B↔7B, Gemma 3 1B↔12B)

  huggingface/trl#7888 · Filter by author meanwo opened on Oct 24, 2026

• The use_reentrant=True override for PEFT + ZeRO-3

  huggingface/trl#7876 · Filter by author cm8908 opened on Oct 23, 2026

• GRPO: with top_p, top_k or min_p, the vLLM importance sampling ratio vanishes and training stalls

  huggingface/trl#7865 · Filter by author albertvillanova opened on Oct 22, 2026

• AsyncGRPO multi-turn tool trajectories can mix policy versions and falsely drop fresh groups

  huggingface/trl#7861 · Filter by author lindicaphxag-tech opened on Oct 21, 2026
  1

• Test every dependency floor at that floor

  huggingface/trl#7853 · Filter by author albertvillanova opened on Oct 21, 2026
  

• Bug: SFTTrainer unconditionally casts QLoRA trainable params to bfloat16, causing GradScaler crash when fp16=True

  huggingface/trl#7851 · Filter by author mrajpurohit1912 opened on Oct 21, 2026
  3