[Feature]: Geometry-aware dispatch to stop the recurring "silent AABB" bug class

[kounelisagis]

Search before asking

• I have searched the issues and discussions and found no similar feature request.

Feature Description

What: A small geometry-aware dispatch layer so detection ops use the geometry each detection actually carries (mask / OBB / box) instead of silently falling back to the axis-aligned xyxy.

Why: Several recent fixes are the same bug in different functions: code grabs detections.xyxy and forgets the real geometry lives in data["xyxyxyxy"] (OBB) or mask. Because xyxy is always present, the wrong answer is the silent default:

• Detections.area returned the AABB area, not the rotated body (fix(detection): make Detections.area OBB-aware #2306)
• with_nms / with_nmm dropped crossed OBBs via AABB IoU (refine(detection): make with_nms and with_nmm OBB-aware #2303)
• as_yolo dropped OBB rotation on export (fix(dataset): preserve OBB rotation in DetectionDataset.as_yolo #2289)
• roboflow/inference: consensus IoU, and stitch / crop corner shifts (fix(workflows): route detections_consensus IoU through detection geometry inference#2420, fix(workflows): shift oriented bounding box corners in detections_stitch inference#2427, fix(workflows): translate oriented bounding box corners in dynamic_crop inference#2430)

Same root cause each time: Detections holds several geometries at once and xyxy is just a derived envelope, so every consumer re-implements "which geometry is the real one?" and forgetting it fails silently. The next geometry-aware function will hit it again.

How: Centralize the dispatch into a few batched helpers and route call sites through them instead of reaching for xyxy. No data-model change, keeps the vectorized design, backward compatible. Migration can be incremental, then deprecate the ad-hoc paths.

Proposed helpers (batched, no per-detection loops):

• detection_iou(a, b): dispatches mask / OBB / box IoU
• detection_area(det): mask sum / shoelace / box area
• move_detections(det, offset): shifts xyxy, mask, OBB corners and keypoints together (what [Bug]: Size-bucketed Precision and F1 count out-of-bucket detections as false positives #2427 / refactor(cv2): add optional backend facade #2430 each forgot)
• a test_op_respects_geometry contract test that fails when an op can't tell an OBB from its envelope

Rollout can be incremental and each step is independent / backward compatible: (1) consolidate the duplicated move_detections (pure dedup, no behavior change), (2) the read-only detection_iou / detection_area dispatch, (3) the contract test.

A first pass suggests more call sites in the same class worth investigating: get_anchors_coordinates (and therefore LineZone / PolygonZone) reads anchors off the envelope rather than the rotated body, as_coco and as_pascal_voc have no is_obb option and drop the rotation on export, with_nmm merges to the union envelope while keeping one input's data["xyxyxyxy"], and DetectionsSmoother averages xyxy without smoothing the oriented corners.

Example Usage

# one dispatch per op, instead of every function re-deriving it
def detection_area(det):
    if det.mask is not None:        return det.mask.sum(axis=(1, 2))
    if "xyxyxyxy" in det.data:      return _shoelace(det.data["xyxyxyxy"])
    return box_area(det.xyxy)

# same shape for detection_iou (mask / OBB / box), plus a geometry-complete
# move_detections that shifts every geometry field together (what #2427/#2430 forgot)

# and a contract test so the class is loud, not silent:
assert op(obb) != op(aabb_of(obb))   # an op must tell an OBB from its envelope

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[suzunn]

I think this would be strongest if the shared helper makes the geometry policy explicit per operation, not just an implicit priority order of mask, OBB, then box. Some call sites genuinely need an envelope output for compatibility, while others should be considered wrong if they fall back to xyxy when richer geometry is present.

A practical shape could be an internal enum or small strategy object such as GeometryMode.EXACT, GeometryMode.ENVELOPE, and GeometryMode.EXPORT_COMPAT. Then the contract tests can use fixtures where two detections have identical envelopes but different OBB or mask geometry. That catches the silent-AABB class directly and also leaves room for legacy exporters to opt into envelope behavior deliberately instead of accidentally.

[kounelisagis]

@suzunn I like the explicit policy, my one worry is keeping it out of the public API. A per operation mode argument adds params across many call sites and a concept every user has to learn, which is a lot for a library this lean.

For almost every op the right policy is just "exact when richer geometry is present", which is already the de facto behavior (0.29.0 made with_nms and area OBB aware). I would keep that as the default and move the decision into one internal geometry resolution helper, so new ops route through it instead of re-deriving from xyxy and silently falling back. EXACT vs ENVELOPE still matters, but as an internal choice at the few ambiguous sites, not a global enum.

For enforcement, alongside your identical envelope fixtures, a metamorphic test fits well: rotate the detections off axis and require an exact op to give the same NMS keep set and area, both rotation invariant on OBB or mask geometry but not on the envelope. One property covers the whole class (masks need a small tolerance since rotation resamples, OBB corners stay exact).

[mvanhorn]

Opened #2374 - a first slice centralizing the geometry-aware dispatch for detection_area/detection_iou behind a single internal resolver, with the metamorphic contract test @kounelisagis suggested. move_detections consolidation and the with_nms/with_nmm/as_yolo migration are left as follow-ups per the incremental rollout.

[Aayan-Ali-Hashim]

Is this issue still open ? I want to contribute to it.

[kounelisagis]

Τook the get_anchors_coordinates slice in #2382

[Shehzad3684]

Hi @kounelisagis! I saw your note about as_coco and as_pascal_voc lacking OBB support and dropping rotation on export. Since #2374 is handling the core area/IoU dispatch and #2382 handled anchors, I would like to take the standalone slice of adding OBB export handling to as_coco and as_pascal_voc. I'll open a focused PR with round-trip unit tests shortly.