Skip to content

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies. #30435

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies.

[Tokenizer] Add tiktoken format loader for OpenAI BPE vocabularies. #30435

Test PJRT plugin  /  Build and test (ubuntu-24.04, cuda)

succeeded Mar 5, 2026 in 3m 5s