feat(coco): read instance-segmentation datasets (polygon→SegmentationMask, identity tracks)

[talmo]

Summary

Adds first-class support for reading COCO instance-segmentation datasets (polygon or RLE) into the segmentation data model, and makes keypoint-free COCO datasets load through the normal auto-detection path. Previously, polygon segmentation was always read as a vector UserROI (only RLE became a SegmentationMask), and load_file() could not auto-detect a COCO file that lacked keypoints.

Key Changes

• Polygon segmentation → SegmentationMask by default. coco.read_labels / read_labels_set / load_coco gain a segmentation_format argument:
  • "mask" (new default): each annotation's polygon(s) are rasterized into a single UserSegmentationMask at the image resolution. Multiple rings of one annotation collapse into one object mask.
  • "roi": keeps the previous behavior (one UserROI per ring).
  • RLE segmentation is always read as a SegmentationMask, regardless of the setting.
• Keypoint-free COCO auto-detection. _is_coco_data now recognizes detection/segmentation-only COCO (a JSON object with images/annotations/categories arrays) instead of requiring keypoints. Label Studio and AlphaTracker exports are JSON arrays, so this dict-shaped signature does not collide with them.
• Categories as identities. A new category_as_track option creates one shared Track per COCO category (named after the category) and assigns it to that category's masks, ROIs, bounding boxes, and keypoint instances (when they have no explicit track id). Useful for instance-segmentation datasets where the category encodes identity rather than object class. Default False.

Example Usage

import sleap_io as sio

# Polygon segmentation -> SegmentationMask (new default), auto-detected even
# without keypoints:
labels = sio.load_file("annotations.coco.json")
masks = labels.labeled_frames[0].masks

# Keep polygons as vector ROIs instead:
labels = sio.load_coco("annotations.coco.json", segmentation_format="roi")

# Treat each category as a persistent identity track:
labels = sio.load_coco("annotations.coco.json", category_as_track=True)
[t.name for t in labels.tracks]                 # one Track per category
labels.labeled_frames[0].masks[0].track.name    # == that mask's category

Masks round-trip through .slp (RLE-compressed mask_rle dataset), with category and identity-track preserved.

API Changes

• coco.read_labels(..., segmentation_format="mask", category_as_track=False) — new keyword args (back-compatible signature; behavior default changed for polygon segmentation, see below).
• coco.read_labels_set(...) and main.load_coco(...) — same two new keyword args, forwarded through; load_file(..., segmentation_format=..., category_as_track=...) forwards them too.
• read_labels raises ValueError for an unknown segmentation_format.

Design Decisions

• Polygon → mask is the new default (breaking). COCO's segmentation field is the segmentation, so mapping it to the SegmentationMask model is the more faithful default and is what exercises the segmentation pipeline end-to-end. The previous polygon→ROI behavior remains available via segmentation_format="roi". The existing polygon-read test was updated and a roi-mode test added.
• Mask rasterization needs image dimensions. Polygons are rasterized at the images entry's height/width. If those are missing, the polygon falls back to an ROI (there is no extent to rasterize into).
• Degenerate polygon rings are skipped. A ring with fewer than 3 vertices (a point/line, or a malformed odd-length ring) cannot form a polygon, so it is skipped rather than crashing Shapely.
• Detection broadening is safe. Only JSON objects with all three COCO arrays match; the array-shaped Label Studio / AlphaTracker formats are unaffected, and detection order is unchanged.

Testing

• New/updated tests in tests/io/test_coco.py: mask-default vs roi-mode reads, multi-polygon→single mask (x and y extents), degenerate-ring skip, dims-fallback, invalid segmentation_format, category_as_track (masks + bboxes + roi-mode ROIs, shared Track objects, .slp round-trip, off-by-default), load_file kwarg forwarding, and a keypoint-free COCO load_file + .slp round-trip.
• New detection test in tests/io/test_main.py for keypoint-free segmentation COCO.
• Reviewed via an adversarial multi-agent pass (multi-dimension review + skeptic verification); confirmed findings applied.
• ruff format/ruff check clean; full test suite green.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.97%. Comparing base (04d0b65) to head (c26d882).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #479   +/-   ##
=======================================
  Coverage   92.96%   92.97%           
=======================================
  Files          54       54           
  Lines       19421    19439   +18     
  Branches     4391     4394    +3     
=======================================
+ Hits        18055    18073   +18     
  Misses        651      651           
  Partials      715      715           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

Docs Preview

Preview has been removed.