Skip to content

AGENTS.md

Latest commit

 

History

History
99 lines (73 loc) · 5.3 KB

AGENTS.md

File metadata and controls

99 lines (73 loc) · 5.3 KB

AGENTS.md

Summary

This is the minimal RGBD point-cloud reference in the repo. Use it when you need the smallest example that aligns stereo depth to a color stream and publishes a point cloud without adding custom host processing or neural inference.

Use This Example When

  • You need a clean baseline for StereoDepth + RGBD + pcl.
  • You want to align depth to RGB or optionally colorize the point cloud from the right mono camera.
  • You need a point-cloud reference that still works with the Visualizer rather than Open3D.
  • You want a simple starting point before moving to measurement or frontend-heavy examples.

Do Not Use This Example When

  • You need box fitting, spatial ROI measurement, or host-side post-processing.
  • You need ToF-specific depth.
  • You need a browser frontend or a desktop GUI.
  • You need neural inference fused into the point-cloud pipeline.

Quick Facts

  • Category: depth-measurement/3d-measurement/rgbd-pointcloud
  • Shape: script+standalone
  • Primary task: align stereo depth to a color source and publish a point cloud
  • Entrypoint: main.py
  • Standalone path: oakapp.toml
  • Frontend: none
  • Runs on: stereo-capable devices with CAM_B/C and either CAM_A or the right mono camera
  • Requires: stereo mono pair, calibration, and optionally RGB camera if --mono is not used
  • Input: stereo depth from CAM_B/C; color from CAM_A by default or right mono with --mono
  • Output: preview and pointcloud
  • Models: none
  • Visualizer / UI: DepthAI Visualizer via dai.RemoteConnection

Read First

Architecture

  • CAM_B and CAM_C feed StereoDepth.
  • RGBD is used to fuse depth with a selected color source.
  • The color source is either:
    • CAM_A in RGB mode
    • the right mono camera when --mono is set
  • RVC4 uses ImageAlign to align depth to the chosen color stream.
  • RVC2 uses stereo.inputAlignTo instead of a separate ImageAlign node.
  • The Visualizer receives both the color preview and the generated point cloud.

Data Flow

  • CAM_B/C -> StereoDepth -> aligned depth
  • CAM_A or right mono -> RGBD.inColor
  • aligned depth + color -> RGBD -> pointcloud
  • selected color stream -> preview

Modification Guide

  • Safe to change: topic names, preview sizing, default color source choice, projector usage
  • Requires care: alignment path differences between RVC2 and RVC4, camera socket assumptions, output frame types
  • Likely to break if changed blindly: switching image size without keeping color/depth alignment consistent, or assuming every device supports an IR projector

Common Adaptations

Constraints

  • This example assumes a stereo mono pair on CAM_B and CAM_C.
  • RGB mode assumes the color camera is on CAM_A.
  • device.setIrLaserDotProjectorIntensity(1) may fail on devices that do not expose that feature.
  • The point cloud is published directly from RGBD; there is no host-side filtering or resampling layer here.

Non-Obvious Repo Conventions

  • preview is the selected color stream, not a rendered depth image.
  • The --mono flag does not disable RGBD; it changes the colorization source to the right mono camera.
  • oakapp.toml provides packaging, but this is still a valid peripheral-mode reference.

Related Examples

Validation

  • Run: python3 main.py
  • Standalone run: oakctl app run .
  • Success looks like: the Visualizer shows preview plus pointcloud, and the point cloud follows the selected color source
  • Common failure meaning: the device lacks the required camera topology, calibration is missing, or RGB/depth alignment assumptions do not match the hardware