AttributeError: Qwen2Tokenizer has no attribute response_schema

[albertvillanova]

As reported by @sergiopaniego, when using transformers dev, we get a 'response_schema' Attribute error for Qwen2Tokenizer:

AttributeError: Qwen2Tokenizer has no attribute response_schema

Reproduction:

from transformers import AutoProcessor

processing_class = AutoProcessor.from_pretrained('Qwen/Qwen3-1.7B')
processing_class.response_schema

See reporting comment:

• CI fails with dev dependencies: ValueError: Could not load tokenizer #4599 (comment)

[albertvillanova]

As reported (#4599 (comment)), it seems the issue appeared after the merge of this PR in transformers:

• rm slow tokenizers transformers#40936

It seems there is an ongoing PR to fix it:

• Fix parse_response after tokenizer refactor transformers#42300

[albertvillanova]

@sergiopaniego, I have locally checked the fixing PR you mentioned (huggingface/transformers#42300), but the problem persists:

processing_class.response_schema
*** AttributeError: Qwen2Tokenizer has no attribute response_schema

[albertvillanova]

Maybe we should replace

processing_class.response_schema

with

getattr(processing_class, "response_schema", None)

After the merge of

• rm slow tokenizers transformers#40936

the attribute response_schema does not necessarily exist. Before it was None.

See comment in PR:

• 🕵️‍♂️ GRPO: Agent training #4300 (review)

[qgallouedec]

Thanks @albertvillanova, response_schema was mistakenly removed and should be added back very soon