refine(detection): make with_nms and with_nmm OBB-aware#2303
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #2303 +/- ##
=======================================
Coverage 80% 80%
=======================================
Files 66 66
Lines 8659 8738 +79
=======================================
+ Hits 6903 6999 +96
+ Misses 1756 1739 -17 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR updates Supervision’s overlap filtering to correctly handle oriented bounding boxes (OBB) in NMS/NMM workflows by routing Detections.with_nms() / with_nmm() through OBB-aware overlap calculations when data[ORIENTED_BOX_COORDINATES] is present. This addresses cases like the “crossed thin rectangles” (X-pattern) where AABB IoU is ~1.0 but true OBB IoU is small, preventing valid detections from being incorrectly suppressed/merged.
Changes:
- Add OBB-aware
oriented_box_non_max_suppression/oriented_box_non_max_mergeand integrate them intoDetections.with_nms()/with_nmm()when oriented box coordinates are present. - Extend
oriented_box_iou_batchwithoverlap_metric(IOU/IOS) and refactor shared greedy NMS/NMM internals into private helpers. - Add regression tests covering the X-pattern for OBB NMS/NMM and an end-to-end
InferenceSlicerregression test for issue #1679.
Review Score (n/5)
- Code quality: 4/5
- Testing: 5/5
- Docs: 4/5
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
src/supervision/detection/utils/iou_and_nms.py |
Adds OBB NMS/NMM APIs, adds overlap_metric to OBB IoU, and refactors shared NMS/NMM helpers. |
src/supervision/detection/core.py |
Routes Detections.with_nms() / with_nmm() through OBB-aware suppression/merge when oriented box data is present. |
src/supervision/__init__.py |
Exports new public OBB NMS/NMM utilities at the top level. |
docs/detection/utils/iou_and_nms.md |
Documents new OBB NMS/NMM utilities on the IoU/NMS docs page. |
tests/detection/utils/test_iou_and_nms.py |
Adds unit tests for overlap_metric support and OBB NMS/NMM behavior (including X-pattern regression). |
tests/detection/tools/test_inference_slicer.py |
Adds end-to-end regression test ensuring SAHI/InferenceSlicer retains crossed OBB detections for both overlap filters. |
tests/detection/test_core.py |
Adds regression tests verifying Detections.with_nms() / with_nmm() dispatch to OBB-aware logic when OBB coordinates exist. |
Comments suppressed due to low confidence (1)
src/supervision/detection/utils/iou_and_nms.py:372
oriented_box_iou_batchnow supports both IoU and IoS viaoverlap_metric, but the docstring still claims it computes “Intersection over Union (IoU)” and returns “Pairwise IoU”, which is inaccurate whenoverlap_metric=IOS. Please update the wording to describe a generic overlap metric / overlap matrix.
Compute Intersection over Union (IoU) of two sets of oriented bounding boxes -
`boxes_true` and `boxes_detection`. Both sets of boxes are expected to be in
`((x1, y1), (x2, y2), (x3, y3), (x4, y4))` format.
Args:
NMS / NMM and OBB metrics call oriented_box_iou_batch with the raw box coordinates, so the rasterization canvas grew with the source image resolution — HD/4K inputs allocated (N, ~4000, ~4000) uint8 masks, easily gigabytes for a few hundred detections. IoU is invariant under translation and uniform scaling, so we shift boxes to the origin (canvas no longer pays for dead space when boxes cluster in a corner) and shrink them when the bounding region exceeds a 1024-pixel cap. Memory is now O(N * 1024^2) regardless of input resolution. Tests cover both transforms via a single parametrized case.
- [resolve roboflow#3] with_nmm: for OBB merge groups with >1 member, set merged xyxy from winner's OBB corners instead of union AABB of all members. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#6] with_nms + with_nmm docstrings: document three-path dispatch order (mask → OBB → AABB) so callers know OBB is now routed differently. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#11] with_nmm: cast OBB to float32 (was float64), aligning with with_nms; relax _group_overlapping_oriented_boxes annotations to np.floating. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested [resolve group] PR roboflow#2303 — items 3, 6, 11 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
- [resolve roboflow#5] oriented_box_non_max_suppression + oriented_box_non_max_merge: add Examples: doctest blocks (project convention for all public APIs). Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#7] oriented_box_iou_batch: update summary/Returns from "Pairwise IoU" to overlap_metric-aware wording, matching box_iou_batch convention. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#8] oriented_box_iou_batch + public entry-points: add ndim/shape validation before reshape to catch flat (N,8) and wrong-ndim inputs early. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#9] oriented_box_non_max_merge: add assert 0 <= iou_threshold <= 1, matching the guard present on oriented_box_non_max_suppression and siblings. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested [resolve group] PR roboflow#2303 — items 5, 7, 8, 9 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
- [resolve roboflow#4] test_with_nmm_falls_back_to_box_nmm_without_obb_data: regression guard that non-OBB Detections still collapse via box NMM path. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - [resolve roboflow#10] boundary tests: empty/single, iou_threshold=0.0/1.0, IOS metric end-to-end NMS, and OBB NMM empty/single — converted threshold test to parametrize(expected_keep) removing if-branch from test body. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested - Added test_with_nmm_obb_merged_xyxy_matches_winner_aabb verifying the item roboflow#3 contract: merged xyxy == winner's AABB, not union of all members. [resolve group] PR roboflow#2303 — items 4, 10 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
…low#2303 - [resolve roboflow#12] add UnReleased entry: with_nms + with_nmm now use OBB IoU when data["xyxyxyxy"] present; callers on old AABB path should strip that key; threshold values may need recalibration. Challenge: evidence=VALID suggestion=VALID resolution=as-suggested [resolve group] PR roboflow#2303 — item 12 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com>
- oriented_box_non_max_suppression: fenced block → >>> doctest, asserts keep == array([ True, False]) - oriented_box_non_max_merge: fenced block → >>> doctest, asserts len(groups) == 1 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
with_nms and with_nmm OBB-awarewith_nms and with_nmm OBB-aware
- TestDetectionsWithNms, TestDetectionsWithNmm in test_core.py - TestOrientedBoxIouBatch, TestOrientedBoxNonMaxSuppression, TestOrientedBoxNonMaxMerge in test_iou_and_nms.py --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
…rize test_drops_true_duplicates + test_is_class_aware → test_suppression_is_class_aware parametrized on (class_id_b, expected_keep): same class suppresses, diff class keeps both --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
- test_respects_overlap_metric → test_overlap_metric_determines_suppression parametrized on (overlap_metric, expected_keep); removes dual-act - TestDetectionsWithNms + TestDetectionsWithNmm shared tests → TestDetectionsObbDispatch parametrized on method name; TestDetectionsWithNmm retains only xyxy-specific test --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Summary
Detections.with_nms/Detections.with_nmmthroughoriented_box_iou_batchwhen detections carrydata["xyxyxyxy"]oriented_box_non_max_suppressionandoriented_box_non_max_merge, exported at the top level and added to the IoU/NMS docs pageoriented_box_iou_batchwithoverlap_metric(IOU/IOS) to match the box variantbox_non_max_suppression/box_non_max_mergeis unchangedInferenceSlicerthat returns 1 detection ondevelopand 2 on this branchCloses #1679.
Why this matters
Two crossed thin OBBs share a near-identical AABB but their oriented bodies barely touch. The current NMS/NMM path uses AABB IoU, so one of them is silently dropped at any reasonable threshold. This is what the reporter hit on YOLOv11x-OBB + SAHI traffic footage. The workarounds suggested in the issue thread (raising
iou_threshold, switching toNON_MAX_MERGE) don't help - the X-pattern AABB IoU is ~1.0 at any threshold below ~1.0. Dispatching on the geometry the user actually provided is the only correct fix.The same indirection affects the
detections_stitchworkflow block inroboflow/inference, which callsmerged.with_nms()internally, so the SAHI workflow there is transitively fixed too.