Skip to content

[data] fix: avoid duplicate BOS after chat template rendering - #170

Open
zhujian19891203 wants to merge 1 commit into
baidu-baige:masterfrom
zhujian19891203:codex/avoid-duplicate-bos-vqa-sft
Open

[data] fix: avoid duplicate BOS after chat template rendering#170
zhujian19891203 wants to merge 1 commit into
baidu-baige:masterfrom
zhujian19891203:codex/avoid-duplicate-bos-vqa-sft

Conversation

@zhujian19891203

@zhujian19891203 zhujian19891203 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • avoid adding special tokens twice after HF chat template rendering in VQA SFT encoding paths
  • tokenize assistant labels with add_special_tokens=False after template rendering
  • apply the same post-template processor behavior to Gr00t N1.6/N1.7 collators

Why

HF chat templates can render BOS directly when called with tokenize=False, for example through {{- bos_token -}} in the template. Passing that rendered prompt to the processor with default special-token handling can prepend another BOS and produce duplicated <|begin_of_text|> in model inputs.

The assistant answer is also tokenized separately to build labels. If that tokenize call adds special tokens, decoded trainable labels can start with BOS even though the answer is not a new document.

This keeps special-token layout owned by the rendered chat template and matches the existing ChatTemplate.encode_multiturn behavior, where text segments are tokenized with add_special_tokens=False.

Example

[decoded_input | colored by role]
<|begin_of_text|><|begin_of_text|><|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
Offer a succinct explanation of the picture presented.
<|vision_start|><|image_pad|>... ...<|image_pad|><|vision_end|><|im_end|>
<|im_start|>assistant
<think>

</think>

7.62x39-HP icon.png<|im_end|>
... ...
[decoded_trainable_labels]
<|begin_of_text|><think>

</think>

7.62x39-HP icon.png<|im_end|>... ...

HF chat templates may render BOS directly when tokenize=False. Passing the rendered prompt to the processor with default special-token handling can prepend another BOS, and separately tokenizing labels can also add BOS to trainable labels. Disable extra special tokens in these post-template encoding paths.
@zhujian19891203 zhujian19891203 changed the title fix(data): avoid duplicate BOS after chat template rendering [data] fix: avoid duplicate BOS after chat template rendering Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant