Skip to content

cuda_mppi_bag_eval.md

Latest commit

 

History

History
247 lines (189 loc) · 9.53 KB

cuda_mppi_bag_eval.md

File metadata and controls

247 lines (189 loc) · 9.53 KB
Raw

CUDA MPPI Bag / Real-Data Evaluation Harness

Reproducible CudaNav Shadow Gate

Use run_cudanav_rosbag_replay.py for release evidence. Unlike the older exploratory runner described below, it refuses a non-empty output directory and fails unless the input bag, selected DB3, controller configuration, diagnostics, evaluation, commands, git commit, clean worktree, and physical GPU identity are cryptographically bound in one manifest.

python3 scripts/run_cudanav_rosbag_replay.py \
  --bag /data/erl_navigation/Prueba2 \
  --evaluation-db /data/erl_navigation/Prueba2/rosbag2_2023_11_13-13_09_49_0.db3 \
  --output-dir build/cudanav_rosbag/erl_prueba2 \
  --controller-config ros2_ws/src/cuda_nav_bringup/config/controller.yaml \
  --controller-command \
    'ros2 launch my_nav shadow_replay.launch.py params_file:={controller_config} diagnostics_csv:={diagnostics_csv}' \
  --profile release \
  --use-sim-time

The controller command is parsed directly into an argument vector; shell operators are not executed. It must consume both {controller_config} and {diagnostics_csv} or the binding gate fails. {out_dir} is also available. Use repeated --bag-play-arg and --record-topic options for platform-specific replay and recording settings.

For bags that already contain compatible sensor_msgs/PointCloud2, TF, and nav_msgs/Path streams, the repository provides a native controller command:

--controller-command \
  'ros2 launch cuda_nav_bringup cudanav_recorded_shadow.launch.py params_file:={controller_config} diagnostics_csv:={diagnostics_csv}'

The native launch runs GPU KISS-ICP, voxel mapping, ESDF, the Nav2 CUDA MPPI controller, and a recorded-Path action adapter. LaserScan-only bags still need an explicit, documented conversion or platform launch; they are not silently accepted as PointCloud2 CudaNav evidence.

The release profile requires:

  • a clean, full git commit and one or more identified NVIDIA GPUs;
  • unchanged source-bag contents and selected DB3 SHA-256;
  • a quality-passing shadow evaluation with at least 60 seconds of recorded motion and 100 CUDA MPPI diagnostic samples;
  • controller config, diagnostics, evaluation, and command-to-input bindings;
  • a content-addressed MCAP output recording with metadata.yaml and storage data;
  • positive recorded message counts for /cuda_nav/cmd_vel, /cuda_nav/odom, /cuda_nav/occupancy, and /cuda_nav/esdf.

The default output recording also retains the CudaNav local map, local costmap, component diagnostics, TF, source points, and recorded plan. Supplying custom --record-topic values in a release run is rejected if any of the four required CudaNav outputs is omitted.

Revalidate the directory, including the external source bag:

python3 scripts/validate_cudanav_rosbag.py \
  build/cudanav_rosbag/erl_prueba2 --profile release

--no-verify-source exists only for inspecting an archived evidence directory after the external dataset has moved. Such a result does not prove that the currently available source bag still matches the run.

This gate deliberately labels the result shadow_controller_with_recorded_motion. Recorded robot motion cannot react to replayed CUDA MPPI commands, so even a passing release-profile replay is not closed-loop navigation evidence.

Exploratory Session Runner

scripts/run_cuda_mppi_bag_eval.py is a thin orchestration wrapper for moving the Nav2 CUDA MPPI controller beyond synthetic maps. It does not assume a particular robot launch file; instead, pass the launch and mission commands used by the target platform.

The harness can:

  • start a Nav2/controller command,
  • play a rosbag2 dataset,
  • optionally send goals or waypoints,
  • record selected topics into a new rosbag2 output,
  • collect cuda_mppi_controller diagnostics CSV,
  • render the diagnostics CSV into SVG/Markdown with scripts/render_cuda_mppi_diagnostics.py.

Typical Bag Replay

python3 scripts/run_cuda_mppi_bag_eval.py \
  --bag /data/site_run \
  --output-dir build/cuda_mppi_bag_eval/site_run \
  --ros-domain-id 101 \
  --use-sim-time \
  --controller-command 'ros2 launch my_nav bringup.launch.py params_file:=/path/to/cuda_mppi.yaml' \
  --mission-command 'python3 scripts/send_waypoints.py --frame map' \
  --duration 120

If the launch file can accept a diagnostics path, use the placeholder:

--controller-command 'ros2 launch my_nav bringup.launch.py diagnostics_csv:={diagnostics_csv}'

The command receives {out_dir} and {diagnostics_csv} placeholders before it is started.

Real Robot / Live Stack

For a live robot, omit --bag and provide --duration:

python3 scripts/run_cuda_mppi_bag_eval.py \
  --output-dir build/cuda_mppi_bag_eval/live_site_a \
  --ros-domain-id 42 \
  --controller-command 'ros2 launch my_robot nav2_cuda_mppi.launch.py' \
  --mission-command 'python3 scripts/send_waypoints.py --file route.yaml' \
  --duration 180

The runner writes:

  • manifest.json with commands, return codes, diagnostics path, and recorded topics,
  • controller.log, mission.log, rosbag_play.log, and rosbag_record.log when those subprocesses are used,
  • topics/ rosbag2 output unless --no-record is set,
  • diagnostics.svg and diagnostics.md if the diagnostics CSV exists.

Controller Config

Set these parameters in the CUDA MPPI controller configuration to collect per-cycle diagnostics:

FollowPath:
  diagnostics_log_period: 1.0
  diagnostics_csv_path: /tmp/cuda_mppi_diagnostics.csv

For repeatable experiments, prefer an output path passed by the harness via {diagnostics_csv} so logs stay with the recorded topics and manifest.

Readout

Use the diagnostics plot to inspect:

  • solve-time spikes against the 20 Hz budget,
  • sustained low valid-rollout ratio,
  • all-colliding and retreat cycles,
  • command saturation or oscillation.

Treat the exploratory runner as an evaluation harness, not a pass/fail benchmark by itself. The scenario, map, localization quality, costmap layers, footprint, and waypoint policy determine whether the run is comparable across commits.

Quality-Gated Offline Report

After a run, combine recorded motion, LaserScan clearance, and optional CUDA MPPI diagnostics into one machine-readable and human-readable result:

python3 scripts/evaluate_mppi_rosbag.py /data/run/topics/run_0.db3 \
  --diagnostics-csv /data/run/diagnostics.csv \
  --output-dir build/mppi_rosbag_evaluation/run

Use --command-topic, --odometry-topic, and --scan-topic when the robot does not use the ERL-inspired topic names. The command writes evaluation.json, evaluation.md, raw clearance samples, decoded motion CSVs, and a per-check CSV. Its default gates cover command pairing, minimum recorded clearance, 50 ms solve p95, valid-rollout ratio, and all-colliding cycles.

The report always records its evidence mode. Diagnostics replayed against a fixed bag are shadow-controller evidence; they are not labeled as closed-loop navigation success.

Public ERL-Inspired Dataset

The open ERL-inspired navigation benchmark dataset contains five ROS 2 Humble runs (6 GB compressed, CC BY 4.0). Download, verify, extract, and inspect its Nav2 topic compatibility with:

python3 scripts/prepare_erl_navigation_bags.py --download --extract

The command writes build/datasets/erl_navigation/compatibility_report.json. Each bag is classified as shadow_ready, adapter_required, or insufficient_for_nav2_replay, and its scan, odometry, TF, command, map, and plan topics are listed. Use the reported topic names to configure remappings before passing a bag to run_cuda_mppi_bag_eval.py.

Recorded motion does not react to newly computed commands. Public bags are therefore suitable for sensor/costmap replay and shadow-mode command analysis, not by themselves for claims about closed-loop controller success.

To avoid downloading the full 6 GB archive, install remotezip, list its members, and extract one selected run with HTTP range requests:

python3 -m pip install remotezip
python3 scripts/prepare_erl_navigation_bags.py --list-remote
python3 scripts/prepare_erl_navigation_bags.py \
  --remote-member Prueba2/rosbag2_2023_11_13-13_09_49_0.db3

The 2026-07-28 recorded-motion result for this run is documented in results/mppi_real_rosbag_erl_prueba2_2026-07-28.md.

Offline inspection without ROS

The rosbag2 SQLite databases can be inventoried on a machine without ROS:

python3 scripts/analyze_rosbag_db3.py /data/erl_navigation/extracted \
  --json build/erl_offline_summary.json \
  --csv build/erl_offline_topics.csv

The outputs report per-bag duration and per-topic message count, observed rate, time coverage, and serialized payload bytes. This inspects recording health and topic availability; it does not deserialize CDR message payloads.

Twist commands and Odometry poses can also be decoded without ROS:

python3 scripts/export_rosbag_motion.py /data/erl_navigation/extracted/Prueba5/*.db3 \
  --output-dir build/prueba5_motion

This writes cmd_vel.csv, odometry.csv, and motion_summary.json, including path length, displacement, observed speeds, command speeds, and stop ratio.

Optional fast-math build

For latency-sensitive, smaller-batch deployments, the CUDA core can be built with -DCUDA_MPPI_FAST_MATH=ON. It is disabled by default because it changes floating-point behavior and may change an MPPI trajectory. See results/mppi_fast_math_2026-07-12.md for the measured speed/quality trade-off and build commands.