You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/concepts/camera_intrinsics.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -159,7 +159,7 @@ K = | 0.0 525.0 239.5 |
159
159
| 0.0 0.0 1.0 |
160
160
```
161
161
162
-
For a different camera or resolution, always supply calibrated values — wrong intrinsics produce correct topology but geometrically distorted metric scale.
162
+
For a different camera or resolution, always supply calibrated values — intrinsics that do not match your camera produce correct topology but geometrically distorted metric scale.
163
163
164
164
See the full field reference and per-entry-point usage examples in the Advanced Config pages:
Copy file name to clipboardExpand all lines: docs/features/depth_estimation.md
+4-35Lines changed: 4 additions & 35 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -114,7 +114,7 @@ Set `VIZION3D_MODEL_CACHE` in your environment to change the default cache direc
114
114
|`return_depth_image`|`bool`| No |`True`| If `True`, the result includes a 16-bit grayscale Open3D Image. Depth Anything V2 outputs inverse relative depth (higher = closer), so higher uint16 values = closer pixels. |
115
115
|`return_raw_depth`|`bool`| No |`True`| If `True`, the result includes the raw depth as a float32 numpy array `(H, W)` — unmodified model output, relative values (not metric). |
116
116
|`return_point_cloud`|`bool`| No |`False`| If `True`, the result includes an Open3D PointCloud unprojected from the RGB-D image. |
117
-
|`advanced_config`|`DepthEstimationAdvanceConfig`| No | PrimeSense defaults | Camera intrinsics and depth range settings. See [Advanced config](#10-advanced-config-camera-intrinsics-depth-range) below. Not sure what intrinsics are? See [Camera Intrinsics Matrix](../concepts/camera_intrinsics.md). |
117
+
|`advanced_config`|`DepthEstimationAdvanceConfig`| No | PrimeSense defaults | Camera intrinsics and depth range settings. See [Advanced config](#8-advanced-config-camera-intrinsics-depth-range) below. Not sure what intrinsics are? See [Camera Intrinsics Matrix](../concepts/camera_intrinsics.md). |
## 9. Advanced config: camera intrinsics & depth range
354
+
## 8. Advanced config: camera intrinsics & depth range
384
355
385
356
`DepthEstimationAdvanceConfig` lets you supply the actual camera intrinsics and depth range for your sensor, replacing the built-in PrimeSense defaults. This is required for accurate metric 3D geometry when your camera is not a 640×480 PrimeSense sensor.
386
357
@@ -410,8 +381,6 @@ The same config is available in the REST and gRPC entry points. See [Advanced Co
410
381
411
382
---
412
383
413
-
---
414
-
415
384
## Known limitations
416
385
417
386
-**Relative depth only** — the default monocular backend produces relative (inverse) depth, not metric depth. Point cloud distances are internally consistent but not calibrated to real-world scale without a known reference distance.
Copy file name to clipboardExpand all lines: docs/features/depth_estimation_advanced_config.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ X = (u - cx) * d / fx
16
16
Y = (v - cy) * d / fy
17
17
```
18
18
19
-
All four intrinsic parameters — `fx`, `fy`, `cx`, `cy` — appear in this formula. Getting them wrong produces a point cloud that is geometrically distorted: correct topology but wrong angles, skewed shapes, or objects that appear compressed or stretched.
19
+
All four intrinsic parameters — `fx`, `fy`, `cx`, `cy` — appear in this formula. Values that do not match your camera produce a point cloud that is geometrically distorted: correct topology but skewed angles, compressed shapes, or stretched geometry.
The vertical focal length in pixels. For cameras with square pixels, `fy ≈ fx`. Cameras with non-square sensors may have `fy ≠ fx`.
46
46
47
-
**Effect on the point cloud:** Controls vertical spread analogously to `fx`. Incorrect`fy` produces vertically compressed or stretched geometry.
47
+
**Effect on the point cloud:** Controls vertical spread analogously to `fx`. A`fy` that does not match your sensor produces vertically compressed or stretched geometry.
48
48
49
49
**How to find it:**`K[1][1]` from the calibration matrix, or:
The horizontal image coordinate of the optical axis — ideally the exact centre of the sensor. For a 640-wide image the ideal value is `319.5`; for a 1920-wide image it is typically near `959.5`.
62
62
63
-
**Effect on the point cloud:** Shifts the entire point cloud left or right. A wrong `cx` makes the scene appear to be viewed from an off-centre vantage point, introducing a lateral tilt.
63
+
**Effect on the point cloud:** Shifts the entire point cloud left or right. A `cx`that does not match your sensor makes the scene appear viewed from an off-centre vantage point, introducing a lateral tilt.
64
64
65
65
---
66
66
@@ -70,7 +70,7 @@ The horizontal image coordinate of the optical axis — ideally the exact centre
70
70
71
71
The vertical image coordinate of the optical axis. For a 480-tall image the ideal value is `239.5`.
72
72
73
-
**Effect on the point cloud:** Shifts the entire point cloud up or down. Like `cx`, an incorrect value introduces a tilt — vertical in this case.
73
+
**Effect on the point cloud:** Shifts the entire point cloud up or down. Like `cx`, a value that does not match your sensor introduces a tilt — vertical in this case.
|**Object at 2.4 m reads as 2.4 m**| Yes — if calibration is correct | No — depends on scene content |
34
-
|**Scale factor to world**| 1.0 (accurate) | Unknown, scene-dependent |
35
-
|**`point_cloud_scale` field**| 1.0 (accurate) | 1.0 (misleading — not real metres) |
35
+
|**Scale factor to world**| 1.0 (real) | Unknown, scene-dependent |
36
+
|**`point_cloud_scale` field**| 1.0 (real metres) | 1.0 (relative, not real metres) |
36
37
|**Shape / topology correct**| Yes | Yes, if correct intrinsics supplied via `DepthEstimationAdvanceConfig`|
37
38
|**Camera calibration needed**| Yes — `focal_length`, `baseline`, `cx`, `cy`| Optional — only affects point cloud geometry |
39
+
|**Compatible with annotation task**| ✅ | ✅ back-projection is self-consistent |
40
+
|**Compatible with other 3D tools**| ✅ registration, reconstruction, metric tools | ⚠️ Z ordering is relative — not directly interoperable with metric clouds |
Copy file name to clipboardExpand all lines: docs/features/stereo_depth.md
+55-32Lines changed: 55 additions & 32 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -118,23 +118,10 @@ curl -L \
118
118
119
119
| Value | What happens |
120
120
|---|---|
121
-
|*(default)*| Downloads the vizion3D release checkpoint (`stereo-depth-s2m2-L.pth`, the L variant) to `~/.cache/vizion3d/models/` on first use, then loads it |
122
-
| An HTTPS URL ending in `.pth` or `.pt`| Downloaded to the cache directory on first use, then loaded as an S2M2 checkpoint |
123
-
| A local `.pth` or `.pt` file path | Loaded directly — no download |
121
+
|*(default)*| Downloads the vizion3D release checkpoint (`stereo-depth-s2m2-L.pth`) to `~/.cache/vizion3d/models/` on first use, then loads it |
124
122
125
123
Models are kept in memory after the first inference. Set `VIZION3D_MODEL_CACHE` to override the cache directory.
126
124
127
-
### S2M2 variants
128
-
129
-
The S2M2 architecture comes in four size variants. The correct one is detected automatically from the checkpoint filename:
Use `scale_factor < 1.0`to downsample input before inference for faster results:
298
+
The handler automatically resizes both images to fit within **960 × 540**before inference, preserving the aspect ratio. This matches the resolution the model was trained near; running at higher resolutions collapses the internal correlation matrix to near-zero disparity and produces an empty point cloud.
312
299
313
-
```python
314
-
from vizion3d.stereo import StereoDepth, StereoDepthAdvancedConfig, StereoDepthCommand
The resize is transparent — disparity and point cloud are reprojected back to the original image dimensions before the result is returned, so all depth values and 3D coordinates are in the original pixel coordinate space. No adjustment to your intrinsics (`focal_length`, `cx`, `cy`) is needed regardless of the input resolution.
|*(input scaling)*| —|automatic|Images are automatically resized to fit within 960×540 before inference, preserving aspect ratio. Disparity and point cloud are reprojected back to the original resolution — metric depth and intrinsics are unaffected. |
A stereo point cloud is in camera space (Z = metric depth, origin at the left camera), making it directly compatible with [Object Mask Annotation 3D](../annotation/object_mask_annotation_3d.md). Pass the same intrinsics you used for stereo depth. Do not pass `image_input` — the annotation task synthesises the segmentation image from the point cloud's stored colours, which avoids having to pick between the left and right frames.
474
+
475
+
```python
476
+
import open3d as o3d
477
+
from vizion3d.stereo import StereoDepth, StereoDepthCommand, StereoDepthAdvancedConfig
478
+
from vizion3d.annotation import ObjectMaskAnnotation3D, ObjectMaskAnnotation3DCommand
479
+
from vizion3d.annotation.models import ObjectMaskAnnotation3DConfig
480
+
481
+
stereo_result = StereoDepth().run(
482
+
StereoDepthCommand(
483
+
left_image="left.png",
484
+
right_image="right.png",
485
+
return_point_cloud=True,
486
+
advanced_config=StereoDepthAdvancedConfig(
487
+
focal_length=1733.74,
488
+
cx=792.27,
489
+
cy=541.89,
490
+
baseline=536.62,
491
+
),
492
+
)
493
+
)
494
+
495
+
annotation_result = ObjectMaskAnnotation3D().run(
496
+
ObjectMaskAnnotation3DCommand(
497
+
point_cloud=stereo_result.point_cloud,
498
+
return_annotated_cloud=True,
499
+
advanced_config=ObjectMaskAnnotation3DConfig(
500
+
fx=1733.74,
501
+
fy=1733.74,
502
+
cx=792.27,
503
+
cy=541.89,
504
+
),
505
+
)
506
+
)
507
+
508
+
for ann in annotation_result.annotations:
509
+
print(f"{ann.label:20s} conf={ann.confidence:.2f} 3D points={len(ann.point_indices)}")
See [Object Mask Annotation 3D — Stereo integration](../annotation/object_mask_annotation_3d.md#5-stereo-point-cloud-integration) for the full walkthrough.
515
+
516
+
---
517
+
495
518
## Known limitations
496
519
497
-
-**Rectified pairs required** — images must be stereo-rectified so corresponding points lie on the same horizontal scanline. Un-rectified pairs will produce incorrect results.
498
-
-**Metric scale depends on calibration** — an incorrect`baseline` or `focal_length` scales all depth values uniformly. Always use calibrated values for real applications.
520
+
-**Rectified pairs required** — images must be stereo-rectified so corresponding points lie on the same horizontal scanline. Un-rectified pairs will not produce reliable results.
521
+
-**Metric scale depends on calibration** — an inaccurate`baseline` or `focal_length` scales all depth values uniformly. Always use calibrated values for real applications.
499
522
-**Python 3.12 required for Open3D** — `return_depth_image` and `return_point_cloud` require Open3D, which currently only supports Python 3.12 in this project.
0 commit comments