Skip to content

Local Harmony tokenizer assigns token ID 200018 to two special tokens #286

Description

@sylvesterkaczmarek

Summary

gpt_oss.tokenizer.get_tokenizer() copies the o200k_base special tokens and then generates every reserved_* token from 200013 through 201087.

o200k_base already assigns ID 200018 to <|endofprompt|>, so the generated <|reserved_200018|> creates a second special-token name for the same ID.

Impact

The tokenizer's special-token mapping is no longer one-to-one. Encoding <|endofprompt|> yields ID 200018, while inverse special-token lookup can resolve that ID to the later reserved alias instead, breaking special-token round-trip expectations and making the local tokenizer inconsistent with a collision-free vocabulary.

The same collision was identified and fixed independently in openai/tiktoken; the local tokenizer implementation in this repository still reproduces the duplicate.

Proposed resolution

When generating Harmony reserved_* entries, skip IDs already occupied by explicit/base special tokens. Add a regression that asserts all special-token IDs are unique and that <|reserved_200018|> is not created while <|endofprompt|> remains mapped to 200018.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions