feat: virtual on-read video cropping (CropVideoBackend) (#460)

Virtual on-read video crops via CropVideoBackend (Video.crop/from_crop), byte-identical to crop_frame, with mosaic shared-decode, SLP round-trip (/video_crops, format 2.3), crop-aware matching, and HDF5 pushdown. Adds apply_crop/apply_crops + 'sio apply-crops' to materialize crops, and imports DLC video_sets crops as provenance (closes #424 item 2). Uncropped files stay byte-identical; old readers degrade gracefully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Author: talmo

docs/cli.md

@@ -72,6 +72,7 @@ sio render --help
 sio trim --help
 sio reencode --help
 sio transform --help
+sio apply-crops --help
 
 # Check version and installed plugins
 sio --version
@@ -2523,6 +2524,26 @@ See the [Transforms Guide](transforms.md#config-file-format) for config file for
 
 ---
 
+## Apply Crops
+
+`sio apply-crops` materializes [virtual crops](cropping.md) (created via `Video.crop` and
+stored in a `.slp`'s `/video_crops`) into real video files, updating the labels to point at
+the baked files. Unlike `sio transform --crop` (which applies a *new* crop and adjusts
+coordinates), this bakes an *existing* virtual crop and is coordinate-neutral.
+
+```bash
+# Bake every virtually-cropped video; baked files go next to the output SLP.
+sio apply-crops mosaic.slp -o baked.slp
+
+# Choose the output video directory and filename suffix.
+sio apply-crops mosaic.slp -o baked.slp --video-dir baked_videos/ --suffix _crop
+```
+
+Each baked video keeps `source_video` provenance to the uncropped original; uncropped videos
+are left untouched. See the [Virtual cropping guide](cropping.md#applying-baking-a-crop-to-disk).
+
+---
+
 ## Use Cases
 
 ### Inspecting an Unknown Labels File

docs/cropping.md

@@ -0,0 +1,219 @@
+# Virtual cropping
+
+sleap-io can expose a **virtual, on-read crop** of a video — a cropped view whose
+frames are produced by decoding the source and slicing in memory, without copying or
+re-encoding any pixels on disk. It is the lazy, non-destructive counterpart of the
+materializing [Transforms](transforms.md) pipeline: a virtually-cropped frame is
+byte-identical to what baking a `Transform(crop=...)` would write.
+
+---
+
+## Quick start
+
+```python
+import sleap_io as sio
+
+full = sio.load_video("session.mp4")              # (1000, 1080, 1920, 3)
+
+# A cropped view. crop = (x1, y1, x2, y2), with x2/y2 EXCLUSIVE.
+view = full.crop((320, 200, 576, 456))
+view.shape            # (1000, 256, 256, 3)  -- cropped
+view[0].shape         # (256, 256, 3)        -- a cropped frame
+view.crop             # not a thing; use view._crop_tuple() -> (320, 200, 576, 456)
+view.source_video is full   # True  -- provenance to the uncropped original
+```
+
+`Video.from_crop` opens a file and crops it in one call:
+
+```python
+view = sio.Video.from_crop("session.mp4", crop=(320, 200, 576, 456))
+```
+
+The returned object is a normal [`Video`](model/video.md): `shape`, `len()`, `grayscale`,
+NumPy-style indexing, and matching all report the **cropped** view.
+
+---
+
+## The crop convention
+
+A crop is `(x1, y1, x2, y2)` in **source pixel coordinates**, with `x2`/`y2`
+**exclusive** — exactly the convention used by [`Transform`](transforms.md) and
+`crop_frame`. The cropped size is `(y2 - y1, x2 - x1)`.
+
+Coordinates may be **negative or extend past the source** — out-of-bounds regions are
+**padded** with `fill` (default `0`), never clamped, so the output shape is always
+exactly `(y2 - y1, x2 - x1)`. This makes fixed-size, centroid-following windows easy:
+
+```python
+# Fixed 128x128 window centered on a point (may run off the frame edge -> padded).
+view = full.crop(center=(cx, cy), size=(128, 128), fill=0)
+view.shape            # (n_frames, 128, 128, 3)
+```
+
+`Video.crop` accepts one region spec — an explicit `crop` rect, a `bbox=(x1,y1,x2,y2)`,
+an `roi` (anything exposing shapely-style `.bounds`, expanded by `margin`), or a
+`center`/`size` pair:
+
+```python
+full.crop((x1, y1, x2, y2))                  # explicit rect
+full.crop(bbox=(x1, y1, x2, y2))             # same, named
+full.crop(roi=my_roi, margin=8)              # axis-aligned bounds of an ROI + margin
+full.crop(center=(cx, cy), size=(w, h))      # fixed-size window
+```
+
+---
+
+## Coordinates
+
+A crop is a pure integer translation by `(x1, y1)`, so mapping landmark coordinates
+between source and cropped frames is exact and NaN-preserving:
+
+```python
+pts_crop   = view.to_crop_coords(pts_source)     # subtract (x1, y1)
+pts_source = view.to_source_coords(pts_crop)      # add (x1, y1)
+```
+
+On an uncropped video these are identity passthroughs, so the same call works
+regardless of whether a video happens to be cropped. The underlying functions live in
+`sleap_io.transform.points` as `crop_points` / `uncrop_points`.
+
+!!! note "Coordinates are never rewritten on disk"
+    Virtual cropping never mutates stored `instance.points`. These helpers are
+    read-time conveniences for presenting/ingesting coordinates in cropped-frame space.
+
+---
+
+## Mosaics: many crops, one decode
+
+Multiple differently-cropped views of one physical file can share a single decoder, so
+the source frame is decoded once per read rather than once per tile:
+
+```python
+full = sio.load_video("session.mp4")
+tiles = [
+    full.crop((x, y, x + 128, y + 128))       # share_decode=True (default)
+    for y in range(0, 1080 - 128, 128)
+    for x in range(0, 1920 - 128, 128)
+]
+labels = sio.Labels(videos=tiles)
+```
+
+Each tile reuses `full`'s backend as its inner reader. The tiles do **not** own that
+shared decoder, so closing one tile does not tear down its siblings; the owning source
+`Video` manages the decoder's lifetime. (Decoder sharing is intentionally not preserved
+across `pickle`/`deepcopy`/`open()` — each reconstruction rebuilds its own reader.)
+
+Two crops of the same file with **different** crops are kept distinct through merge,
+append, and matching; two crops with the **same** rect dedup to one view.
+
+---
+
+## Saving & loading (SLP round-trip)
+
+Crops round-trip through `.slp` without breaking older readers:
+
+```python
+sio.save_file(labels, "mosaic.slp")
+labels2 = sio.load_file("mosaic.slp")
+labels2.videos[0]._crop_tuple()         # (0, 0, 128, 128)        -- preserved
+labels2.videos[0].shape                 # (1000, 128, 128, 3)
+labels2.videos[0].source_video.shape    # (1000, 1080, 1920, 3)
+len(labels2.videos)                     # all tiles preserved (not collapsed)
+```
+
+- The crop rects are stored in a dedicated top-level `/video_crops` dataset, written
+  **only when a crop is present**; the `videos_json` entry describes the **uncropped
+  source**.
+- An older reader that does not understand `/video_crops` simply loads the uncropped
+  source video — a graceful, lossy degrade, never an error.
+- Files with no crops are byte-identical to before this feature existed (no
+  `/video_crops`, no format-version bump).
+
+---
+
+## Applying (baking) a crop to disk
+
+A virtual crop can be **materialized** to a real video file — the cropped pixels become
+physical and the crop is no longer a read-time view. This is coordinate-neutral: a virtual
+crop already presents cropped-frame coordinates, so baking the pixels leaves all point
+coordinates unchanged.
+
+`Video.apply_crop` bakes one cropped video and returns a new `Video` for the baked file,
+preserving provenance (`source_video` is the uncropped original):
+
+```python
+view = full.crop((320, 200, 576, 456))
+baked = view.apply_crop("crop.mp4")
+baked.shape                    # (1000, 256, 256, 3)  — cropped, now physical
+baked.source_video.shape       # (1000, 1080, 1920, 3) — uncropped original
+baked._crop_tuple()            # None — the crop is materialized, not virtual
+```
+
+`Labels.apply_crops` bakes every virtually-cropped video in a `Labels` and rewires all
+references (labeled frames, ROIs, suggestions) to the baked files; uncropped videos are
+untouched and coordinates are unchanged:
+
+```python
+labels.apply_crops(video_dir="baked_videos/")   # one file per tile, unique names
+```
+
+From the command line, `sio apply-crops` materializes every virtual crop in an SLP,
+writing baked videos to a directory next to the output and updating the references:
+
+```bash
+sio apply-crops mosaic.slp -o baked.slp --video-dir baked_videos/
+```
+
+!!! note "`apply_crop` vs `sio transform --crop`"
+    `apply_crop` materializes an **existing** virtual crop (no coordinate change).
+    `sio transform --crop` applies a **new** crop and adjusts coordinates — that is the
+    materializing [`transform_video`](transforms.md) / `transform_labels` path:
+
+    ```python
+    sio.transform_video(full, "baked.mp4", sio.Transform(crop=(320, 200, 576, 456)))
+    ```
+
+!!! info "Encoder padding"
+    The H.264 encoder pads frame dimensions up to a multiple of 16 (bottom/right only,
+    preserving the top-left content and coordinate alignment). A baked video whose cropped
+    width/height are not multiples of 16 is padded on those edges.
+
+---
+
+## Performance expectations
+
+The crop is applied **after** a full-frame decode for every backend except raw,
+sub-frame-chunked HDF5, where it can push the region read down to the storage layer:
+
+| Backend | Strategy | I/O effect |
+|---|---|---|
+| `MediaVideo` (mp4/H.264/…) | decode full frame, slice | **No decode/I/O savings** — inter-frame codecs must decode the whole frame; the slice is a free in-memory view. Saves resident array size only. |
+| `HDF5Video` raw rank-4, **sub-frame chunked** | hyperslab region read (`ds[i, y1:y2, x1:x2, :]`) | **Real I/O reduction** — only the overlapping chunks are read/decompressed. The one case where a crop saves disk work. |
+| `HDF5Video` raw rank-4, per-frame chunked | region read (whole chunk still fetched) | Modest — skips chunk reassembly, not I/O. |
+| `HDF5Video` embedded PNG/JPEG (`.pkg.slp`) | decode full image, slice | **No savings** — the whole image must be decoded before any spatial selection. |
+| `ImageVideo`, `TiffVideo`, `SeqVideo` | decode full frame, slice | **No savings** with the current decoders. |
+
+Pushdown for raw HDF5 is automatic and gated on the dataset's actual chunking; it falls
+back to a full decode plus slice (byte-identical) whenever it would not help.
+
+---
+
+## Non-goals
+
+Virtual cropping is a pure translate-and-clip view. It deliberately does **not** do:
+
+- **Rotation, scale, pad, or flip on read** — those remain the domain of the
+  materializing [`Transform`](transforms.md) pipeline.
+- **Decode-cost savings for compressed video** — only sub-frame-chunked raw HDF5 sees
+  real I/O savings; everywhere else the crop is a free post-decode view.
+- **Lossless export through non-SLP writers** (NWB, COCO, JABS, Ultralytics) — those
+  formats have no crop concept; exporting a cropped `Labels` through them is acceptably
+  lossy (the cropped frame and its coordinates are emitted as-is).
+- **Rewriting on-disk point coordinates** — the source labels are never mutated.
+
+---
+
+## See also
+
+- [Transforms](transforms.md): the materializing crop/scale/rotate/pad/flip pipeline.
+- [Video](model/video.md): the `Video` facade and its backends.

docs/examples.md

@@ -1007,6 +1007,73 @@ sio.save_video(sio.load_video("input.mp4"), "output.mp4")
 !!! note "See also"
     [`save_video`](formats/#sleap_io.save_video): Video saving options and codec settings
 
+### Virtual cropping and batch autocrop
+
+Expose a virtual, on-read crop of a video — frames are decoded and sliced in memory, with no pixels copied or re-encoded ([`Video.crop`](model/video.md#sleap_io.Video.crop) / [`Video.from_crop`](model/video.md#sleap_io.Video.from_crop)). The crop is `(x1, y1, x2, y2)` in source pixels (`x2`/`y2` exclusive); out-of-bounds regions are padded.
+
+```python title="virtual_crop.py" linenums="1"
+import sleap_io as sio
+
+full = sio.load_video("session.mp4")              # (1000, 1080, 1920, 3)
+view = full.crop((320, 200, 576, 456))            # virtual view, no decode yet
+view.shape                                         # (1000, 256, 256, 3)
+view.is_cropped, view.crop_rect                    # True, (320, 200, 576, 456)
+view.source_video is full                          # True - provenance preserved
+frame = view[0]                                    # decode-then-slice (256, 256, 3)
+
+# Other region specs: a bbox, an ROI (+ margin), or a fixed-size centered window.
+view = full.crop(bbox=(320.0, 200.0, 576.0, 456.0))
+view = full.crop(roi=my_shapely_poly, margin=8)
+view = full.crop(center=(cx, cy), size=(128, 128))   # fixed shape; off-frame is padded
+```
+
+**Batch autocrop (e.g. a multi-chamber rig).** Apply a fixed set of per-chamber rects across many recordings and write one cropped file per `(video x chamber)`. `apply_crop` bakes the virtual crop to disk and keeps `source_video` pointing at the uncropped original.
+
+```python title="batch_autocrop.py" linenums="1"
+import sleap_io as sio
+from pathlib import Path
+
+# Chamber layout, defined once (x1, y1, x2, y2). 16-aligned dims avoid encoder padding.
+chambers = {
+    "A": (0, 0, 640, 480),
+    "B": (640, 0, 1280, 480),
+    "C": (0, 480, 640, 960),
+    "D": (640, 480, 1280, 960),
+}
+
+out_dir = Path("crops")
+out_dir.mkdir(exist_ok=True)
+for path in Path("recordings").glob("*.mp4"):
+    full = sio.load_video(path.as_posix())
+    for name, rect in chambers.items():
+        crop = sio.Video.from_crop(full, rect)
+        crop.apply_crop((out_dir / f"{path.stem}_{name}.mp4").as_posix())
+```
+
+Prefer to stay lazy (no re-encode) and carry the crops in a labels file? Build the views into a `Labels`, save (crops ride a `/video_crops` dataset; pixels are untouched), and bake them all later in one call with [`Labels.apply_crops`](model/labels.md#sleap_io.Labels.apply_crops):
+
+```python title="virtual_crop_slp.py" linenums="1"
+import sleap_io as sio
+
+full = sio.load_video("session.mp4")
+tiles = [sio.Video.from_crop(full, rect) for rect in chambers.values()]
+sio.save_file(sio.Labels(videos=tiles), "session.slp")   # virtual; no re-encode
+
+# Later - materialize every virtual crop to real files and update references:
+sio.load_file("session.slp").apply_crops(video_dir="crops/")
+```
+
+The same step is available from the command line for an SLP that already carries virtual crops:
+
+```bash
+sio apply-crops session.slp -o baked.slp --video-dir crops/
+```
+
+!!! note "See also"
+    - [Virtual cropping guide](cropping.md): conventions, mosaics, coordinates, performance, and non-goals.
+    - [`Video.apply_crop`](model/video.md#sleap_io.Video.apply_crop) / [`Labels.apply_crops`](model/labels.md#sleap_io.Labels.apply_crops): materialize virtual crops to disk.
+    - [Transforms](transforms.md): the materializing crop/scale/rotate/pad/flip pipeline (`sio transform --crop` applies a *new* crop and adjusts coordinates).
+
 ### Switch video and image backends
 
 Control which backend is used for video reading and embedded frame encoding.

docs/formats/dlc.md

@@ -16,17 +16,32 @@ up from the CSV — the following extra metadata is imported:
 Pass `config=False` to disable config use entirely and reproduce the legacy,
 config-free output.
 
-!!! note "Cropping is not yet applied"
+!!! note "Cropping (`video_sets[...].crop`)"
     DeepLabCut's `video_sets[...].crop` is a *virtual* read-time crop (an ROI
-    that DLC's video reader slices out of each full frame on the fly). When a
-    project uses cropping, the images under `labeled-data/<video>/` are the
-    cropped region and the labels are stored in **cropped-frame coordinates**,
-    whereas the linked `source_video` points at the original, **uncropped**
-    video. sleap-io does not yet apply this crop, so for cropped projects the
-    labels are offset from the source video by the crop origin `(x1, y1)`.
-    Reconciling the two requires virtual ROI-cropping of a `Video` on read,
-    which is planned future work. For the common case of no cropping (the DLC
-    default is the full frame), there is no offset and the link is exact.
+    that DLC's video reader slices out of each full frame). The images under
+    `labeled-data/<video>/` are the cropped region and the labels are stored in
+    **cropped-frame coordinates**, while the linked `source_video` is the
+    original, **uncropped** video. sleap-io now imports this crop:
+
+    - The crop rect is parsed from `video_sets` (DLC stores it width-range-first
+      as `x1, x2, y1, y2`; sleap-io reorders it to its `(x1, y1, x2, y2)`
+      convention, `x2`/`y2` exclusive) and recorded under
+      `labels.provenance["dlc_crops"]`, keyed by source-video path. This record
+      **persists through an SLP round-trip**.
+    - Labels are left **verbatim in cropped-frame coordinates** on the uncropped
+      `labeled-data` `ImageVideo` — no offset is applied (and the already-cropped
+      images are never cropped again). To map a label into the full source frame,
+      use [`Video.to_source_coords`](../model/video.md#sleap_io.Video.to_source_coords)
+      with the recorded rect (it adds the crop origin `(x1, y1)`).
+    - When the source video file is available, `source_video` is set to a
+      [`Video.from_crop`](../model/video.md#sleap_io.Video.from_crop) view of it,
+      so `source_video.crop_rect` / `to_source_coords` work in memory (this view's
+      crop is in-memory only; the persistent record is `provenance["dlc_crops"]`).
+      When the source is absent, `source_video` is a closed `Video` as before.
+    - Identity crops at the origin (`0, W, 0, H` — the DLC no-cropping default)
+      record no crop and leave the link exact.
+
+    See the [virtual cropping guide](../cropping.md) for the crop conventions.
 
 ```python
 import sleap_io as sio