Dataset Summary from Dataset class

[hardikdava]

An idea is to extend Dataset class to get summary of whole dataset. Something like,

• Total numbers of labels and images
• Numbers of labels per class
• Class imbalance identification
• Crowded scenario by finding maximum iou
• Finding average nms iou threshold based on dataset

[SkalskiP]

Hi, @hardikdava 👋🏻 ! I like the idea a lot! Would you like to work on it? If so, let's start with scoping this and defining the API.

[SkalskiP]

⚠️ API definition is still in progress

Here is the initial draft of the Dataset Summary API

Overall direction

• Similarly, as with Detections API, we aim for ease of use and flexibility
• ease of use - offer a set of ready-to-use functions
• flexibility - allow custom query execution

Proposed code structure

@dataclass
class ImageSummary
    name: np.ndarray                # store image names
    width: np.ndarray
    height: np.ndarray
    aspect_ratio: np.ndarray

@dataclass
class DetectionSummary
    image_name: np.ndarray          # store image names
    class_id: np.ndarray
    class_name: np.ndarray
    x: np.ndarray
    y: np.ndarray
    width: np.ndarray
    height: np.ndarray

Supported statistics

• total images
• total annotations
• per class annotation count

example of that stat calculated in Roboflow

• per image annotation count

example of that stat calculated in Roboflow

Proposed API

@dataclass
class ImageSummary
    name: np.ndarray                # store image names
    width: np.ndarray
    height: np.ndarray
    aspect_ratio: np.ndarray

@dataclass
class DetectionSummary
    image_name: np.ndarray          # store image names
    class_id: np.ndarray
    class_name: np.ndarray
    x: np.ndarray
    y: np.ndarray
    width: np.ndarray
    height: np.ndarray

@dataclass
class DetectionDatasetSummary
    pass

Proposed usage

>>> import supervision as sv

>>> ds = sv.DetectionDataset.from_yolo(...)

>>> summary = ds.describe()

>>> type(summary)

DetectionDatasetSummary

[hardikdava]

@SkalskiP Looks nice overall 😍.

[SkalskiP]

Thanks! 🔥 I still need to think about it. It will be super powerful feature when we release it.

[hardikdava]

@SkalskiP this might be interesting for us.

[SkalskiP]

Hi, @hardikdava 👋🏻!

Yup. I noticed two competing projects:

• https://github.com/Deci-AI/data-gradients
• https://github.com/lancedb/yoloexplorer

We will probably need to do some tests and research. Look for strong and weak sides. And make it better.

[hardikdava]

Yeah, I already tested data-gradients and I am going to test yoloexplorer today in the evening.

[hardikdava]

@SkalskiP , my thoughts based on experimenting both framework.

yoloexplorer

• Highly depends on ultralytics package so more like ultralytics specific
• Poor visualization
• Heavy framework needed e.g. pytorch
• More features might be added in future since just started recently.
• Currently, only dataset profiling and analysis providing

Data-Gradient

• Highly depends on super-gradient package so more like deci ai specific
• Good insights of dataset even per image detailing
• Heavy framework needed e.g. pytorch data loaders
• Currently, only dataset profiling and analysis providing

supervision

• Only depends on numpy, opencv , no pytorch
• Can work with pytorch or tensorflow or keras_cv or super-gradient
• More dataset operations like merge, remove, etc
• More dataset conversion possible (no one provides this)

[hardikdava]

Found something similar feature : https://github.com/laclouis5/globox/tree/main

[hardikdava]

Some useful features be like finding issues in image such as blurred images, dark and bright images, etc.

[SkalskiP]

The question is: Can we do it without pulling more advanced dependencies?

[hardikdava]

Yeah, i already did it in the past with only using opencv and numpy operations.

[SkalskiP]

@hardikdava I think we should gradually add more things but if you made it work in the past we can definitely make it part of the scope.