Add YOLO26 monocular depth estimation#305
Conversation
Wire a new depth task end to end, mirroring the semantic pipeline. The exported ONNX outputs a single-channel float depth map in meters at input resolution; postprocessing strips the letterbox padding and bilinear-resizes to the original image size. Results expose a DepthMap matching the Python results.depth API, and annotation renders an INFERNO-colorized map side by side with the original image. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
The fast_image_resize F32 path was single-threaded here, costing ~8.7ms per image and dominating GPU inference. Replace it for depth with a rayon-parallel bilinear over output rows (the same pattern as the semantic slow path), which also matches Python's align_corners=False F.interpolate exactly. Postprocess drops to ~0.8ms with identical depth values. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Both the depth resize and the semantic slow-path recomputed the same align_corners=False half-pixel index math inline. Extract a small bilinear_axis helper and call it from both, removing the duplicated formula. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Drop the 256-entry colormap table in favor of a 6th-order polynomial fit of OpenCV's COLORMAP_INFERNO (within ~9/255 per channel). Same visual result for depth maps, far less code, no lookup table. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Extract the duplicated per-column bilinear LUT into bilinear_x_lut, used by both the depth and semantic resamplers. The depth same-resolution fast path is removed: the general bilinear path already produces an exact copy at scale 1. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Add a depth field to the JS payload: an opaque INFERNO-colorized RGBA image plus the min/max range in meters, colorized with the shared visualizer colormap so browser depth matches native. Depth already flows through the reused core postprocess, so no task dispatch changes are needed. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Expose depth (RGBA overlay) and depth_range on the Results type and draw the colorized depth map in annotate, mirroring the mask overlay path. Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
|
All Contributors have signed the CLA. ✅ |
|
👋 Hello @onuralpszr, thank you for submitting a
For more guidance, please refer to our Contributing Guide. This is an automated message, and an engineer will assist with any follow-up questions. Thank you for contributing to Ultralytics! 🚀 |
UltralyticsAssistant
left a comment
There was a problem hiding this comment.
🔍 PR Review
Made with ❤️ by Ultralytics Actions
RuntimeError
Debug Info
Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 561, in generate_pr_review
response = get_agent_response(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 476, in get_agent_response
next_input = list(pool.map(lambda call: _handle_function_call(call, tool_handlers), function_calls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 476, in <lambda>
next_input = list(pool.map(lambda call: _handle_function_call(call, tool_handlers), function_calls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 365, in _handle_function_call
output = tool_handlers[name](**_parse_tool_arguments(call))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 145, in read_file
text = _read_head_file(event, head_sha, local_checkout, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 111, in _read_head_file
return _fetch_head_file(event, head_sha, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 95, in _fetch_head_file
raise RuntimeError(f"fetch failed for {path} at PR head: HTTP {response.status_code}")
RuntimeError: fetch failed for src/metadata.rs at PR head: HTTP 502
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Closing to re-push with re-signed (GPG-verified) commits; a filter-branch earlier stripped signatures from some commits. Reopening a fresh PR from the same branch. |
Adds monocular depth estimation (YOLO26) as a first-class task across the whole stack, mirroring the semantic segmentation path. The exported ONNX/TFLite output a single-channel float depth map in meters; postprocessing strips the letterbox padding and bilinear-resizes to the original image, and results expose a
DepthMapthat matches the Pythonresults.depthAPI. Visualization renders the original beside an INFERNO-colorized depth map (a polynomial fit of OpenCV's colormap, so no lookup table), and all five sizes (yolo26{n,s,m,l,x}-depth) auto-download. Depth values were validated against the PythonDepthPredictorand agree within rounding on every size.The browser package gets the same task: pre/postprocessing stay in Rust/wasm and results carry a colorized depth overlay plus the min/max range, so depth runs in the browser via both ONNX Runtime Web and LiteRT.js and produces the same values as native.
Requires the Python-side depth work in ultralytics/ultralytics#25065 to merge first (the
.pt/ONNX/TFLite exports and the traceable calibration this builds on), tracked here so the two stay in sync.🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
🧭 Adds YOLO26 monocular depth estimation support across the Rust inference library, CLI, WebAssembly bindings, and TypeScript API.
📊 Key Changes
Depthtask andDepthMapresults type, providing per-pixel depth values in meters.-depthmodel discovery and automatic downloading, including models such asyolo26n-depth.onnx.--task depth, saving outputs underruns/depth/.🎯 Purpose & Impact
-depthONNX exports.