Merge pull request #44 from OlafenwaMoses/main

release - ScaleObservation task - estimation by object inference and reference + Scene mask annotation

Author: OlafenwaMoses

.gitignore

@@ -236,5 +236,9 @@ image.png
 image2.png
 sample.py
 
+# Research workspace and generated research outputs are intentionally excluded
+# from product lint/format checks and version control.
+research/
+
 # uv python pin (library — consumers choose their own Python)
 .python-version

docs/CNAME

@@ -0,0 +1 @@
+docs.vizion3d.org

docs/annotation/object_mask_annotation_3d.md

@@ -173,7 +173,10 @@ The default checkpoint (`yolo26l-seg.pt`) is trained on the [COCO](https://cocod
 | **Appliances** | microwave, oven, toaster, sink, refrigerator |
 | **Indoor** | book, clock, vase, scissors, teddy bear, hair drier, toothbrush |
 
-Objects not in this list will not be detected. To annotate other categories, supply a custom YOLO segmentation checkpoint via `model_backend`.
+Objects not in this list will not be detected by the default checkpoint. To
+annotate other categories, supply a custom YOLO segmentation checkpoint via
+`model_backend`, or use the prompt-free YOLOE model described below for broader
+annotation coverage.
 
 ---
 
@@ -226,8 +229,8 @@ Each `MaskAnnotation3D` item contains:
 
 | Field | Type | Description |
 |---|---|---|
-| `label` | `str` | COCO class name, e.g. `"person"`, `"chair"`. |
-| `class_id` | `int` | COCO integer class index (0-based). |
+| `label` | `str` | Model class name, e.g. `"person"`, `"chair"`. With the default checkpoint this is a COCO class name; with prompt-free YOLOE this is a YOLOE runtime label. |
+| `class_id` | `int` | Model class index (0-based). With the default checkpoint this is the COCO class index. |
 | `confidence` | `float` | Detection confidence in `[0, 1]`. |
 | `bbox_2d` | `list[float]` | Bounding box in image pixels: `[x1, y1, x2, y2]`. |
 | `mask_2d` | `np.ndarray` | Boolean segmentation mask, shape `(H, W)`. |
@@ -590,3 +593,51 @@ result = ObjectMaskAnnotation3D().run(
 - **Relative depth point clouds** — if the input point cloud was generated by monocular depth estimation (which produces relative, not metric, depth), object sizes in 3D will not correspond to real-world dimensions. For metric results, use a calibrated stereo or RGB-D camera.
 - **Open3D required** — this task requires Open3D, which currently only supports Python 3.12 in this project.
 - **Front-view synthesis** — when no image is supplied, the synthesised view is a simple point-splatting projection. Dense regions render well; sparse or occluded regions may produce a patchy image that reduces detection quality compared to a real photo.
+
+## Prompt-free YOLOE model for broader annotation
+
+`ObjectMaskAnnotation3D` also supports the prompt-free YOLOE segmentation
+checkpoint released with the essentials bundle:
+
+```text
+https://github.com/OlafenwaMoses/vizion3D/releases/download/essentials-v1/yoloe-26l-seg-pf.pt
+```
+
+The model can be passed directly through `model_backend`; no annotation-library
+code changes are required:
+
+```python
+annotation_result = ObjectMaskAnnotation3D().run(
+    ObjectMaskAnnotation3DCommand(
+        point_cloud=pcd,
+        image_input="scene.jpg",
+        model_backend=(
+            "https://github.com/OlafenwaMoses/vizion3D/releases/download/"
+            "essentials-v1/yoloe-26l-seg-pf.pt"
+        ),
+    )
+)
+```
+
+This prompt-free model exposes **4,585 runtime labels**, so it is useful when
+the goal is broad instance annotation beyond the default COCO categories.
+
+Recommended use:
+
+- Use the default `yolo26l-seg.pt` checkpoint when COCO coverage is sufficient
+  and higher closed-set COCO segmentation accuracy is preferred.
+- Use `yoloe-26l-seg-pf.pt` when broad prompt-free annotation is more important
+  than COCO-only precision.
+- For ScaleObservation, `yoloe-26l-seg-pf.pt` can provide additional useful
+  labels such as desks, cabinets, monitors, doors, and appliances. Most of its
+  4,585 labels still do not have calibrated metric priors, so unsupported labels
+  are ignored by the scale estimator. See
+  [Scale Observation](../observation/scale_observation.md#supported-scale-priors)
+  for the current prior list.
+
+The supported prompt-free labels are listed and grouped here:
+
+- [YOLOE-26L Prompt-Free Supported Classes](./yoloe_26l_prompt_free_classes.md)
+
+The source runtime label files used to build that class reference are also kept
+under `research/instance_segmentation/`.