Dfine just exploding

[FrancescoSaverioZuppichini]

Hi there :)

Trying to run the dfine training notebook without any change and it exploding during training

ValueError: Caught ValueError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
           ^^^^^^^^^^^^^^^^^^^^
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
            ~~~~~~~~~~~~^^^^^
  File "/tmp/ipykernel_10044/1942330765.py", line 58, in __getitem__
    transformed = self.transform(image=image, bboxes=boxes, category=categories)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/composition.py", line 607, in __call__
    self.preprocess(data)
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/composition.py", line 647, in preprocess
    self._preprocess_processors(data)
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/composition.py", line 674, in _preprocess_processors
    processor.preprocess(data)
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/utils.py", line 256, in preprocess
    data[data_name] = self.check_and_convert(data[data_name], shape, direction="to")
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/bbox_utils.py", line 340, in check_and_convert
    self.check(converted_data, shape)
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/bbox_utils.py", line 355, in check
    check_bboxes(data)
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/augmentations/utils.py", line 245, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/media/zuppif/Cancro/open-omniparser-dataset/omniparser/.venv/lib/python3.12/site-packages/albumentations/core/bbox_utils.py", line 625, in check_bboxes
    raise ValueError(f"x_max is less than or equal to x_min for bbox {invalid_bbox}.")
ValueError: x_max is less than or equal to x_min for bbox [1.         0.41666666 1.         1.         3.        ].

The error seems to say that the width must be bigger than one, in this case it isn't.

Do you have any idea on what is going on over there? Thanks a lot!

[FrancescoSaverioZuppichini]

Also one comment is the code says

    @staticmethod
    def format_image_annotations_as_coco(image_id, categories, boxes):
        """Format one set of image annotations to the COCO format

        Args:
            image_id (str): image id. e.g. "0001"
            categories (List[int]): list of categories/class labels corresponding to provided bounding boxes
            boxes (List[Tuple[float]]): list of bounding boxes provided in COCO format
                ([center_x, center_y, width, height] in absolute coordinates) <--- HERE

but that's YOLO format, coco format as the top coordinates of the top left corner to my knowledge

[merveenoyan]

@FrancescoSaverioZuppichini I ran the notebook and ran into no issues. bbox format is ok, for training it uses (x_min, y_min, x_max, y_max) for internal processing it uses center format. Docstring is wrong though, you're right about it.

Here's the logs:

Can you confirm so we can close the issue?

[FrancescoSaverioZuppichini]

@FrancescoSaverioZuppichini I ran the notebook and ran into no issues. bbox format is ok, for training it uses (x_min, y_min, x_max, y_max) for internal processing it uses center format. Docstring is wrong though, you're right about it.

Here's the logs:

Can you confirm so we can close the issue?

Thanks a lot.

So that is not COCO, are you 100% sure about it?

My dataset is not similar to what it was pretrained and it has just one class. I think I've not made mistakes since I can plot the bboxes with no issue using hf code but again I am using COCO; I have to change a little bit the map metric because in the case on just one class it was exploding for me.

In my experiments the model with the hf code doesn't learn. I am using this repo atm and it trains perfectly, faster compared than hf (no overhead of the Trainer probably) and with no issue and very good convergence. Let me try to run the dataset in the original notebook as well to compare