Skip to content

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies. #30435

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies.

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies. #30435

Test TensorFlow  /  Linux (x86_64)

succeeded Mar 5, 2026 in 1m 16s