Video object tracking via SAM 2 / SAM 3 (tracking issue)

[wkentaro]

What to build

Tracking issue for video object tracking using SAM 2 / SAM 3 memory across frames, so labelme's video/multi-frame annotation can propagate an object across frames instead of re-annotating each frame. SAM 3 (released 2025-11) adds promptable concept segmentation and tracking across video; the underlying models are already reachable via osam.

This is a large bet and a HITL tracking issue: it needs design (frame model, propagation UX, storage) before implementation slices are cut.

Acceptance criteria

• Design captured: how tracked objects propagate across frames and how results are stored
• Implementation broken into smaller slices once the design is agreed

Blocked by

None - design discussion can start immediately