Numpy DataItem

[eugene123tw]

Summary

How to test

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[ashwinvaidya17]

Thanks for the efforts. It looks good overall. I have a few minor comments

[ashwinvaidya17]

Thanks! I am fine with the changes

[kprokofi]

Thanks for the update, Eugene. That said, I feel now the NumpyDataItem is starting to overcomplicate things. We're now duplicating logic by preprocessing and collating everything separately into NumPy arrays. On top of that, for IR validation, we still need the annotations as PyTorch tensors since we're using torchmetrics for evaluation.

Would it make sense to support only NumPy images with optional transforms, and keep the annotations in torch format? From the perspective of the OV Engine and model implementations, it seems more consistent that way.
What do you think?

[eugene123tw]

Thanks for the update, Eugene. That said, I feel now the NumpyDataItem is starting to overcomplicate things. We're now duplicating logic by preprocessing and collating everything separately into NumPy arrays. On top of that, for IR validation, we still need the annotations as PyTorch tensors since we're using torchmetrics for evaluation.

Would it make sense to support only NumPy images with optional transforms, and keep the annotations in torch format? From the perspective of the OV Engine and model implementations, it seems more consistent that way. What do you think?

@kprokofi I think we should avoid depending on torchmetrics for IR inference/validation. Using torchmetrics requires converting both predictions and annotations into PyTorch tensors, which doesn’t make sense for IR tasks — the outputs are typically generic (e.g., lists, NumPy arrays) and shouldn't be tied to a specific deep learning framework.

I also feel mixing NumPy arrays and torch tensors within a single data entity isn’t clean and could introduce bugs or unnecessary complexity.

A more framework-agnostic alternative could be HuggingFace’s evaluate library (https://huggingface.co/docs/evaluate/index), which supports segmentation, object detection, and other tasks without depending on any particular DL framework.

As for the duplication concern: if we remove torchmetrics dependency for IR, we wouldn’t need to collate annotations separately into tensors — we could simply pass datasets with NumPy data items directly to the evaluator, without needing to wrap them in a PyTorch DataLoader at all. This would simplify the flow.

What do you think?