Skip to content

Add {% generation %} markers for Cohere2 chat template#5675

Merged
qgallouedec merged 7 commits intomainfrom
cohere2-training
May 4, 2026
Merged

Add {% generation %} markers for Cohere2 chat template#5675
qgallouedec merged 7 commits intomainfrom
cohere2-training

Conversation

@qgallouedec
Copy link
Copy Markdown
Member

@qgallouedec qgallouedec commented Apr 28, 2026

see #5471


Note

Medium Risk
Moderate risk because it changes template auto-swapping logic for Cohere2 and adds a new Jinja template that affects how prompts are rendered during SFT/GRPO, which could subtly impact tokenization and loss masking for those models.

Overview
Adds Cohere2 to TRL’s list of recognized chat-template families and introduces cohere2.jinja plus a patched cohere2_training.jinja that wraps assistant output in {% generation %} / {% endgeneration %} (and moves <|END_OF_TURN_TOKEN|> into role branches) to enable correct assistant_only_loss masking.

Updates get_training_chat_template() to detect the Cohere2 original template and swap in the new training variant, and extends the chat-template test matrix to cover the Cohere2 tokenizer; docs/README are updated to document Cohere2 support and the patch rationale.

Reviewed by Cursor Bugbot for commit 0d9bcc0. Bugbot is set up for automated code reviews on this repo. Configure here.

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a115d9df13

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread trl/chat_templates/cohere2_training.jinja
@qgallouedec
Copy link
Copy Markdown
Member Author

merging this with no review: it's been opened for a while an nothing special/potentially breaking

@qgallouedec qgallouedec merged commit 0a3d956 into main May 4, 2026
12 of 13 checks passed
@qgallouedec qgallouedec deleted the cohere2-training branch May 4, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants