You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/api/lifting.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,7 @@ Input contract for the depth estimation task. All inference parameters are decla
22
22
23
23
## DepthEstimationAdvanceConfig
24
24
25
-
Camera intrinsics and depth range settings. Pass an instance of this model as `advanced_config` on `DepthEstimationCommand` to override the PrimeSense defaults used for point cloud unprojection.
25
+
Camera intrinsics and depth range settings. Pass an instance of this model as `advanced_config` on `DepthEstimationCommand` to override the PrimeSense defaults used for point cloud unprojection. See [Camera Intrinsics Matrix](../concepts/camera_intrinsics.md) for a full explanation of `fx`, `fy`, `cx`, and `cy`.
The camera intrinsics matrix **K** (also called the calibration matrix) encodes the optical properties of a camera that map 3D world points onto a 2D image plane. Every point cloud produced by vizion3d is built from these four numbers.
4
+
5
+
---
6
+
7
+
## The matrix
8
+
9
+
```
10
+
| fx 0 cx |
11
+
K = | 0 fy cy |
12
+
| 0 0 1 |
13
+
```
14
+
15
+
In full projection form — taking a 3D point `(X, Y, Z)` in camera coordinates and projecting it to image pixel `(u, v)`:
16
+
17
+
```
18
+
| u | | fx 0 cx | | X/Z |
19
+
| v | = | 0 fy cy | × | Y/Z |
20
+
| 1 | | 0 0 1 | | 1 |
21
+
```
22
+
23
+
Expanded:
24
+
25
+
```
26
+
u = fx × (X / Z) + cx
27
+
v = fy × (Y / Z) + cy
28
+
```
29
+
30
+
Inverted (what vizion3d does when building a point cloud from a depth map):
31
+
32
+
```
33
+
Z = d
34
+
X = (u - cx) × d / fx
35
+
Y = (v - cy) × d / fy
36
+
```
37
+
38
+
where `d` is the depth value at pixel `(u, v)`.
39
+
40
+
---
41
+
42
+
## Each element explained
43
+
44
+
### `fx` — horizontal focal length (pixels)
45
+
46
+
Position in K: row 0, col 0.
47
+
48
+
The horizontal focal length is the product of the physical focal length of the lens and the horizontal pixel density of the sensor. It describes how strongly the camera compresses horizontal depth into horizontal pixel distance.
49
+
50
+
-**Large `fx`** → narrow horizontal field of view; objects appear wider in pixel space.
51
+
-**Small `fx`** → wide horizontal field of view; objects appear narrower in pixel space.
52
+
53
+
Relation to horizontal field of view `FoV_h`:
54
+
55
+
```
56
+
fx = (image_width / 2) / tan(FoV_h / 2)
57
+
```
58
+
59
+
Read from a calibration matrix: K\[0\]\[0\].
60
+
61
+
---
62
+
63
+
### `fy` — vertical focal length (pixels)
64
+
65
+
Position in K: row 1, col 1.
66
+
67
+
The vertical focal length. For cameras with square pixels `fy ≈ fx`. Non-square sensors (rare in modern hardware) have `fy ≠ fx`.
68
+
69
+
-**Large `fy`** → narrow vertical field of view.
70
+
-**Small `fy`** → wide vertical field of view.
71
+
72
+
Relation to vertical field of view `FoV_v`:
73
+
74
+
```
75
+
fy = (image_height / 2) / tan(FoV_v / 2)
76
+
```
77
+
78
+
Read from a calibration matrix: K\[1\]\[1\].
79
+
80
+
---
81
+
82
+
### `cx` — horizontal principal point (pixels)
83
+
84
+
Position in K: row 0, col 2.
85
+
86
+
The x-coordinate of the **optical axis** on the image sensor — the pixel column where a ray travelling straight through the centre of the lens hits the sensor. Ideally the exact horizontal centre of the image.
87
+
88
+
- For a **640-wide** image: `cx ≈ 319.5`
89
+
- For a **1920-wide** image: `cx ≈ 959.5`
90
+
91
+
A miscalibrated `cx` shifts the entire point cloud left or right, making the scene appear viewed from an off-centre position.
92
+
93
+
Read from a calibration matrix: K\[0\]\[2\].
94
+
95
+
---
96
+
97
+
### `cy` — vertical principal point (pixels)
98
+
99
+
Position in K: row 1, col 2.
100
+
101
+
The y-coordinate of the optical axis on the sensor. Ideally the exact vertical centre of the image.
102
+
103
+
- For a **480-tall** image: `cy ≈ 239.5`
104
+
- For a **720-tall** image: `cy ≈ 359.5`
105
+
106
+
A miscalibrated `cy` shifts the entire point cloud up or down.
107
+
108
+
Read from a calibration matrix: K\[1\]\[2\].
109
+
110
+
---
111
+
112
+
### Off-diagonal zeros and the bottom row
113
+
114
+
```
115
+
| fx 0 cx |
116
+
K = | 0 fy cy |
117
+
| 0 0 1 |
118
+
```
119
+
120
+
-**K\[0\]\[1\] = 0** — the skew coefficient. Zero for all modern digital cameras (pixels are rectangular).
|**Camera datasheet**| Look for focal length in mm and sensor pixel pitch; `fx = f_mm / pixel_size_mm`. |
135
+
|**Field of view approximation**|`fx = (W/2) / tan(FoV_h/2)`, `cx = W/2 − 0.5`. Accurate enough for a first test. |
136
+
137
+
---
138
+
139
+
## Using K in vizion3d
140
+
141
+
Supply the four values via `advanced_config` on any depth or stereo command:
142
+
143
+
```python
144
+
from vizion3d.lifting import DepthEstimationAdvanceConfig
145
+
146
+
config = DepthEstimationAdvanceConfig(
147
+
fx=909.15, # K[0,0]
148
+
fy=908.48, # K[1,1]
149
+
cx=640.0, # K[0,2]
150
+
cy=360.0, # K[1,2]
151
+
)
152
+
```
153
+
154
+
Without `advanced_config`, vizion3d defaults to the **PrimeSense / Kinect v1** intrinsics at 640×480:
155
+
156
+
```
157
+
| 525.0 0.0 319.5 |
158
+
K = | 0.0 525.0 239.5 |
159
+
| 0.0 0.0 1.0 |
160
+
```
161
+
162
+
For a different camera or resolution, always supply calibrated values — wrong intrinsics produce correct topology but geometrically distorted metric scale.
163
+
164
+
See the full field reference and per-entry-point usage examples in the Advanced Config pages:
@@ -38,9 +110,10 @@ Set `VIZION3D_MODEL_CACHE` in your environment to change the default cache direc
38
110
|---|---|---|---|---|
39
111
|`image_input`|`str \| bytes`|**Yes**| — | Image to process. Pass a file path string or raw image bytes. |
40
112
|`model_backend`|`str`| No | vizion3D release checkpoint URL | Model backend identifier. See [Model backends](#model-backends) above. |
41
-
|`return_depth_image`|`bool`| No |`False`| If `True`, the result includes a 16-bit grayscale Open3D Image of the depth map. |
113
+
|`return_depth_image`|`bool`| No |`True`| If `True`, the result includes a 16-bit grayscale Open3D Image. Depth Anything V2 outputs inverse relative depth (higher = closer), so higher uint16 values = closer pixels. |
114
+
|`return_raw_depth`|`bool`| No |`True`| If `True`, the result includes the raw depth as a float32 numpy array `(H, W)` — unmodified model output, relative values (not metric). |
42
115
|`return_point_cloud`|`bool`| No |`False`| If `True`, the result includes an Open3D PointCloud unprojected from the RGB-D image. |
43
-
|`advanced_config`|`DepthEstimationAdvanceConfig`| No | PrimeSense defaults | Camera intrinsics and depth range settings. See [Advanced config](#10-advanced-config-camera-intrinsics-depth-range) below. |
116
+
|`advanced_config`|`DepthEstimationAdvanceConfig`| No | PrimeSense defaults | Camera intrinsics and depth range settings. See [Advanced config](#10-advanced-config-camera-intrinsics-depth-range) below. Not sure what intrinsics are? See [Camera Intrinsics Matrix](../concepts/camera_intrinsics.md). |
44
117
45
118
---
46
119
@@ -54,7 +127,8 @@ Set `VIZION3D_MODEL_CACHE` in your environment to change the default cache direc
54
127
|`min_depth`|`float`| Yes | Minimum value in `depth_map`. |
55
128
|`max_depth`|`float`| Yes | Maximum value in `depth_map`. Guaranteed `max_depth >= min_depth`. |
56
129
|`backend_used`|`str`| Yes | Resolved model identifier that processed the request (local file path). |
57
-
|`depth_image`|`open3d.geometry.Image \| None`| When `return_depth_image=True`| 16-bit grayscale image, dtype `uint16`, shape `(H, W)`. The full 0–65535 range maps to `[min_depth, max_depth]`. |
|`raw_depth`|`np.ndarray \| None`| Yes (set `return_raw_depth=False` to suppress) | Float32 array, shape `(H, W)`. Raw model output — relative values, not metric. |
58
132
|`point_cloud`|`open3d.geometry.PointCloud \| None`| When `return_point_cloud=True`| Coloured 3D point cloud unprojected from the RGB-D image using the intrinsics in `advanced_config`. Coordinates are in metres. |
59
133
|`point_cloud_scale`|`float`| Yes | Scale factor: multiply any distance measured between two points in the point cloud by this value to get the equivalent distance in metres. Always `1.0` — Open3D produces point cloud coordinates directly in metres. |
Copy file name to clipboardExpand all lines: docs/features/depth_estimation_advanced_config.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,8 @@
6
6
7
7
## Background: the pinhole camera model
8
8
9
+
> Not sure what `fx`, `fy`, `cx`, `cy` are? See the [Camera Intrinsics Matrix](../concepts/camera_intrinsics.md) reference for a full explanation of the K matrix and how to read it for your camera.
10
+
9
11
Every point in a point cloud is computed by inverting the pinhole camera projection. Given a pixel at image coordinates `(u, v)` with a depth value `d` (in metres), its 3D position `(X, Y, Z)` is:
0 commit comments