Skip to content

DETRv2 training notebook produces bad model #10

@matheger

Description

@matheger

Trying to run through https://github.com/qubvel/transformers-notebooks/blob/main/notebooks/RT_DETR_v2_finetune_on_a_custom_dataset.ipynb at commit 89a48c3

The trained model performs much worse than the output cells suggest. Best detection score in the last inference step is ~0.2 or less.

I had to change a few things to make this notebook work (these shouldn't really affect the model):

  1. Drop three examples with bad bounding boxes from dataset["train"] (indices 65, 383, 476)
  2. Fix the category issue from here
  3. Set dataloader_num_workers=0, otherwise they keep crashing (multiprocessing issues?)
  4. Save to/load from disk instead of pushing to hub

My environment:

python 3.11
transformers 4.56.2
torch 2.8.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions