Skip to content

Supporting class-specific boxes in CombinedNonMaxSuppression op #1765

Open
@masahi

Description

@masahi

Currently, CombinedNonMaxSuppression converter does not support class-specific boxes, i.e. q == num_class case in https://www.tensorflow.org/api_docs/python/tf/image/combined_non_max_suppression.

utils.make_sure(share_boxes_across_classes,
"CombinedNonMaxSuppression only currently implemented for boxes shared across classes.")

Apparently, "official" TF FasterRCNN / MaskRCNN models uses such variant of combined nms. So we cannot convert them to ONNX right now.
https://github.com/tensorflow/models/blob/238922e98dd0e8254b5c0921b241a1f5a151782f/official/vision/detection/modeling/architecture/heads.py#L360-L361

I've hit this issue when trying to convert QAT-ed MaskRCNN model from https://github.com/openvinotoolkit/nncf/tree/develop/examples/tensorflow/segmentation. The model is attached for repro.
https://drive.google.com/file/d/1vaNmyKH0C0n5PGlo2oZ2wggME1Blg5sf/view?usp=sharing

I wonder if it is possible to encode this variant using a combination of ONNX NMS + other ops. For example, by making boxes into the shape (batch, num_boxes * num_class, 4) and doing "one hot" encoding for scores. Ideally we could change the ONNX spec to allow this variant, since the implementation is actually trivial (just need to change indexing into boxes).

@TomWildenhain-Microsoft

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions