Skip to content

Retire three tiny model generators with no test consumer - #7186

Open
albertvillanova wants to merge 3 commits into
mainfrom
retire-unused-tiny-model-generators
Open

Retire three tiny model generators with no test consumer#7186
albertvillanova wants to merge 3 commits into
mainfrom
retire-unused-tiny-model-generators

Conversation

@albertvillanova

@albertvillanova albertvillanova commented Sep 11, 2026

Copy link
Copy Markdown
Member

Three tiny models are generated and published but loaded by no test. This PR drops their generators.

Found by cross-referencing every push_to_hub call in the 64 generator scripts against the model ids appearing under tests/.

Tiny model Status
tiny-Qwen2ForCausalLM-2.5-Coder never referenced under tests/, in any commit
tiny-Qwen3MoeForSequenceClassification was a tokenizer source only; #5469 replaced those uses with tiny-Qwen3MoeForCausalLM
tiny-PaliGemmaForConditionalGeneration added to the DPO vision list already commented out in #2287; #4728 removed the dead comment

Why this is worth doing

An unused tiny model is not free. It is regenerated on every alignment pass, and each regeneration costs a Hub PR to open, review and merge, with no test that would notice if the result were broken.

It also makes the pre-merge check for a tiny model PR silently vacuous. apply_model_revisions in tests/conftest.py injects a revision only when a test loads that exact id, so a MODEL_REVISIONS entry for an unused model never fires: the suite goes green whether or not the regenerated artefact is sound. That is what happened on #7134, where the entry would have had no effect and the run was skipped.

Questions

CC: @huggingface/trl

Delete or revive? Deleting is the claim that these three models are not worth covering. The opposite reading is defensible, particularly for PaliGemma: it is a prefix-LM VLM, a shape no other tiny model in the suite has, and it was disabled from the moment it was added with no reason recorded, unlike the Idefics2 line above it which cites a transformers issue. If you would rather revive one, say which and I will drop that commit and open the coverage work separately. There is one commit per model, so any of the three can be dropped on its own.

What about the Hub repos? This PR only removes the generators. The repositories under trl-internal-testing stay as they are. Options are to leave them, archive them, or delete them. Leaving them means an id that still resolves, with no script left to regenerate it, which is the state that produced this cleanup in the first place. I would rather not touch the org without a decision from you.

Changes

  • Drop for_causal_lm/qwen2_for_causal_lm_2_5_coder.py
  • Drop for_sequence_classification/qwen3_moe_for_sequence_classification.py
  • Drop for_conditional_generation/paligemma_for_conditional_generation.py

Note

Low Risk
Deletes maintenance-only scripts with no test consumers; no runtime or library behavior changes.

Overview
Removes three unused tiny-model generator scripts under scripts/generate_tiny_models/: Qwen2.5-Coder causal LM, Qwen3 MoE sequence classification, and PaliGemma conditional generation. Each script built a shrunk config, ran smoke checks, and pushed to Hub via push_to_hub, but no test in the suite loads those published ids anymore.

This stops regenerating and opening Hub alignment PRs for models that nothing in CI would exercise. Hub repos are unchanged—only the local generators are deleted.

Reviewed by Cursor Bugbot for commit 0ed090d. Bugbot is set up for automated code reviews on this repo. Configure here.

`trl-internal-testing/tiny-Qwen2ForCausalLM-2.5-Coder` is not loaded by any
test, and never has been: the id has never appeared under `tests/` in any
commit. The script was carried into `scripts/generate_tiny_models/` by the
split in #5637 from the single-file list added in #2287.
`trl-internal-testing/tiny-Qwen3MoeForSequenceClassification` was only ever a
tokenizer source in the chat template tests. #5469 replaced those uses with
`tiny-Qwen3MoeForCausalLM`, which is the natural source for a Qwen3-MoE
tokenizer, and left the generator behind with no consumer.
`trl-internal-testing/tiny-PaliGemmaForConditionalGeneration` has never run.
#2287 added it to the DPO vision trainer list already commented out, and #4728
removed the dead comment. No test has loaded it since.
@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant