Skip to content

Add YOLO26 monocular depth estimation#305

Closed
onuralpszr wants to merge 11 commits into
mainfrom
depth-anything-rust-onnx
Closed

Add YOLO26 monocular depth estimation#305
onuralpszr wants to merge 11 commits into
mainfrom
depth-anything-rust-onnx

Conversation

@onuralpszr

@onuralpszr onuralpszr commented Jul 10, 2026

Copy link
Copy Markdown
Member

Adds monocular depth estimation (YOLO26) as a first-class task across the whole stack, mirroring the semantic segmentation path. The exported ONNX/TFLite output a single-channel float depth map in meters; postprocessing strips the letterbox padding and bilinear-resizes to the original image, and results expose a DepthMap that matches the Python results.depth API. Visualization renders the original beside an INFERNO-colorized depth map (a polynomial fit of OpenCV's colormap, so no lookup table), and all five sizes (yolo26{n,s,m,l,x}-depth) auto-download. Depth values were validated against the Python DepthPredictor and agree within rounding on every size.

The browser package gets the same task: pre/postprocessing stay in Rust/wasm and results carry a colorized depth overlay plus the min/max range, so depth runs in the browser via both ONNX Runtime Web and LiteRT.js and produces the same values as native.

Requires the Python-side depth work in ultralytics/ultralytics#25065 to merge first (the .pt/ONNX/TFLite exports and the traceable calibration this builds on), tracked here so the two stay in sync.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

🧭 Adds YOLO26 monocular depth estimation support across the Rust inference library, CLI, WebAssembly bindings, and TypeScript API.

📊 Key Changes

  • ➕ Introduces the Depth task and DepthMap results type, providing per-pixel depth values in meters.
  • 🧠 Adds YOLO26-only -depth model discovery and automatic downloading, including models such as yolo26n-depth.onnx.
  • 🔄 Implements depth post-processing with letterbox cropping and bilinear resizing to restore the original image dimensions.
  • 🎨 Adds INFERNO color mapping for depth visualization.
  • 🖼️ Updates image annotation to generate an original-plus-colorized-depth side-by-side output.
  • 💻 Extends the CLI with --task depth, saving outputs under runs/depth/.
  • 🌐 Exposes colorized RGBA depth overlays and depth ranges through the WebAssembly and TypeScript APIs.
  • 📚 Updates English and Chinese documentation, examples, task tables, and API references.
  • ✅ Adds unit and integration tests covering depth parsing, resizing, summaries, visualization, model downloading, and end-to-end inference.

🎯 Purpose & Impact

  • 🚀 Enables Rust, CLI, browser, and WebAssembly users to run YOLO26 depth estimation alongside existing detection and segmentation tasks.
  • 📏 Provides depth results at the original image resolution, making them easier to consume in downstream applications.
  • 🎥 Supports practical visualization through colorized depth maps for images, video, and web canvases.
  • 🔌 Maintains an Ultralytics-compatible results structure, simplifying migration and multi-task application development.
  • ⚠️ Depth estimation is currently limited to YOLO26 models and requires compatible -depth ONNX exports.

onuralpszr added 10 commits July 9, 2026 09:13
Wire a new depth task end to end, mirroring the semantic pipeline. The
exported ONNX outputs a single-channel float depth map in meters at input
resolution; postprocessing strips the letterbox padding and bilinear-resizes
to the original image size. Results expose a DepthMap matching the Python
results.depth API, and annotation renders an INFERNO-colorized map side by
side with the original image.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
The fast_image_resize F32 path was single-threaded here, costing ~8.7ms per
image and dominating GPU inference. Replace it for depth with a rayon-parallel
bilinear over output rows (the same pattern as the semantic slow path), which
also matches Python's align_corners=False F.interpolate exactly. Postprocess
drops to ~0.8ms with identical depth values.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Both the depth resize and the semantic slow-path recomputed the same
align_corners=False half-pixel index math inline. Extract a small bilinear_axis
helper and call it from both, removing the duplicated formula.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Drop the 256-entry colormap table in favor of a 6th-order polynomial fit of
OpenCV's COLORMAP_INFERNO (within ~9/255 per channel). Same visual result for
depth maps, far less code, no lookup table.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Extract the duplicated per-column bilinear LUT into bilinear_x_lut, used by
both the depth and semantic resamplers. The depth same-resolution fast path is
removed: the general bilinear path already produces an exact copy at scale 1.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Add a depth field to the JS payload: an opaque INFERNO-colorized RGBA image
plus the min/max range in meters, colorized with the shared visualizer colormap
so browser depth matches native. Depth already flows through the reused core
postprocess, so no task dispatch changes are needed.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
Expose depth (RGBA overlay) and depth_range on the Results type and draw the
colorized depth map in annotate, mirroring the mask overlay path.

Signed-off-by: Onuralp SEZER <onuralp@ultralytics.com>
@github-actions

Copy link
Copy Markdown

All Contributors have signed the CLA. ✅

@UltralyticsAssistant UltralyticsAssistant added documentation Improvements or additions to documentation enhancement New feature or request labels Jul 10, 2026
@UltralyticsAssistant

Copy link
Copy Markdown
Member

👋 Hello @onuralpszr, thank you for submitting a ultralytics/inference 🚀 PR! To ensure a seamless integration of your YOLO26 monocular depth estimation feature, please review the following checklist:

  • Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • Synchronize with Source: Confirm your PR is synchronized with the ultralytics/inference main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • Update Documentation: Update the relevant documentation for any new or modified features.
  • Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
  • Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. This is an automated message, and an engineer will assist with any follow-up questions. Thank you for contributing to Ultralytics! 🚀

@UltralyticsAssistant UltralyticsAssistant left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

⚠️ Review generation encountered an error: RuntimeError

Debug Info
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 561, in generate_pr_review
    response = get_agent_response(
               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 476, in get_agent_response
    next_input = list(pool.map(lambda call: _handle_function_call(call, tool_handlers), function_calls))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 476, in <lambda>
    next_input = list(pool.map(lambda call: _handle_function_call(call, tool_handlers), function_calls))
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/utils/openai_utils.py", line 365, in _handle_function_call
    output = tool_handlers[name](**_parse_tool_arguments(call))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 145, in read_file
    text = _read_head_file(event, head_sha, local_checkout, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 111, in _read_head_file
    return _fetch_head_file(event, head_sha, path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/actions/review_pr.py", line 95, in _fetch_head_file
    raise RuntimeError(f"fetch failed for {path} at PR head: HTTP {response.status_code}")
RuntimeError: fetch failed for src/metadata.rs at PR head: HTTP 502

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.15419% with 11 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/postprocessing.rs 96.33% 4 Missing ⚠️
src/results.rs 90.90% 4 Missing ⚠️
src/annotate.rs 92.10% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@onuralpszr

Copy link
Copy Markdown
Member Author

Closing to re-push with re-signed (GPG-verified) commits; a filter-branch earlier stripped signatures from some commits. Reopening a fresh PR from the same branch.

@onuralpszr onuralpszr closed this Jul 10, 2026
@onuralpszr onuralpszr deleted the depth-anything-rust-onnx branch July 10, 2026 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants