Depth Estimation

[brianzheng206]

We want to integrate the Depth Anything v2 model into a ROS node for real-time depth estimation. The model will be pre-compiled into a TensorRT engine file for high-performance inference on NVIDIA GPUs. The ROS node should subscribe to an image topic, perform depth inference, and publish the resulting depth map as either a sensor_msgs/Image or PointCloud2 (configurable).

The main objective is to achieve close to real time speeds; our old depth estimation node was too slow (running inference on pytorch models) and not making use of our hardware

Goals:
Convert Depth Anything v2 repo to a TensorRT engine file (FP16 or INT8 quantization if feasible).

Build a ROS node to:

• Subscribe to a configurable image topic (RGB camera feed).
• Run inference through TensorRT.
• Publish the depth map in a ROS-compatible format.
• Optionally publish derived data (point cloud, disparity).
• Optimize for throughput and low latency (target: ≥30 FPS)

[lucasreljic]

Use this existing repo for converting to TensorRT; it should be a cookie-cutter approach.
depth-anything-tensorrt

[thomasonzhou]

Depth Anything V2 predicts affine-invariant depth right? Not metric

[lucasreljic]

Yes, but there is also a metric depth fine-tuned model here metric_depth for Depth Anything V2. Although it might be a good idea to research and implement some type of depth completion approach like this Zero-shot Depth Completion.