Skip to content

Reject a zero length in the FixedLength pre-tokenizer instead of panicking#2155

Open
v-code01 wants to merge 1 commit into
huggingface:mainfrom
v-code01:fixedlength-reject-zero
Open

Reject a zero length in the FixedLength pre-tokenizer instead of panicking#2155
v-code01 wants to merge 1 commit into
huggingface:mainfrom
v-code01:fixedlength-reject-zero

Commits

Commits on Jul 5, 2026