I am using data-gradients for object-detection data analysis. but end up with the following error. Is this caused by different number of objects in each image? I am using Yolo-format label file.

[krishiv26]

💡 Your Question

File "analyse_dataset.py", line 27, in
analyzer.run()
File "/usr/local/lib/python3.8/dist-packages/data_gradients/managers/abstract_manager.py", line 226, in run
self.execute()
File "/usr/local/lib/python3.8/dist-packages/data_gradients/managers/abstract_manager.py", line 114, in execute
for i, (train_batch, val_batch) in enumerate(datasets_tqdm):
File "/usr/local/lib/python3.8/dist-packages/tqdm/std.py", line 1195, in iter
for obj in iterable:
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 635, in next
data = self._next_data()
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 679, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
return self.collate_fn(data)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 143, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 143, in
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 172, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 120, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/collate.py", line 163, in collate_tensor_fn
return torch.stack(batch, 0, out=out)
RuntimeError: stack expects each tensor to be equal size, but got [1, 5] at entry 0 and [4, 5] at entry 2

Versions

[pip3] numpy==1.22.2
[pip3] pytorch-quantization==2.1.2
[pip3] torch==1.14.0a0+44dac51
[pip3] torch-tensorrt==1.4.0.dev0
[pip3] torchtext==0.13.0a0+fae8e8c
[pip3] torchvision==0.15.0a0
[pip3] triton==2.0.0

[Louis-Dupont]

Hi @krishiv26 could you please share the snippet of code you are using?

[DarshnaZE]

Hi @Louis-Dupont I am facing the same issue. and i am following the https://github.com/Deci-AI/data-gradients/blob/e468899d2368296ad7515439eca9524fbe38718e/examples/example_detection_super_gradients.py

When working with batch_size=1 , it works perfectly fine, faces the issue while increasing the batch_size

[Louis-Dupont]

@DarshnaZE you run the script as is and got the error ? Also which version of DG are you using ?
I just tried with the master branch and it all worked without adding batch_size.

[DarshnaZE]

@Louis-Dupont Thanks. i tried with removing the batch_size param. it works same as when set batch_size=1. It takes around 3 hrs for around 200k images, as it runs on cpu (if i am not wrong). so increasing the batch_size will help decreasing the analysis time.