Add cross-library TSDF / ESDF / Occupancy benchmark drivers - #287
Draft
fwilliams wants to merge 1 commit into
Draft
Add cross-library TSDF / ESDF / Occupancy benchmark drivers#287fwilliams wants to merge 1 commit into
fwilliams wants to merge 1 commit into
Conversation
Add a `tests/benchmarks/tsdf_fusion/` benchmark suite that compares
fvdb against the public sparse-voxel libraries (nvblox, VDBFusion,
Open3D) on real and synthetic RGB-D / LiDAR workloads. Pairs with the
fvdb-core `feat/tsdf-esdf-stack` PR.
What's included
---------------
Per-dataset bench drivers under `cross_library/`:
bench_open3d_vs_fvdb.py Synthetic sphere scene, TSDF only.
Self-contained smoke test (no external
dataset download).
bench_replica_full.py Replica multi-scene, TSDF + ESDF at
full frame rate.
bench_esdf_replica.py Replica ESDF scale sweep.
bench_kitti.py KITTI Odometry LiDAR TSDF.
bench_esdf_kitti.py KITTI Odometry LiDAR ESDF.
bench_occupancy_kitti.py KITTI Odometry LiDAR occupancy.
bench_decay_kitti.py KITTI Odometry dynamic-scene decay.
bench_mai_city.py Mai City LiDAR TSDF, fvdb vs VDBFusion
vs nvblox.
bench_seven_scenes.py 7-Scenes long-trajectory TSDF.
bench_esdf_vs_nvblox.py Mai City ESDF, fvdb vs nvblox.
bench_occupancy_vs_nvblox.py Mai City occupancy, fvdb vs nvblox.
Supporting modules:
kitti_loader.py, replica_loader.py, mai_city_loader.py,
seven_scenes_loader.py
Minimal per-dataset loaders that yield
`(intrinsics, cam_to_world, depth_or_points)` tuples.
download_kitti.py, download_replica.py, download_replica_zip.py
Resumable, parallel-stream downloaders for the public dataset
archives.
nvblox_runner.py
Python wrapper around the nvblox CLI that matches the cross-
library interface used by the bench drivers above.
install_nvblox.sh, install_vdbfusion.sh
Reproducible install recipes for the two C++ comparison
libraries (each builds against the fvdb conda env's TBB /
Blosc / Boost).
data/README.md, data/.gitignore
Conventions for placing dataset trees under `data/`; the dataset
files themselves are not vendored.
How to use
----------
Each driver accepts `--help` for its full argument surface; typical
invocations are documented in the per-driver module docstrings. The
drivers write a JSON results file (path configurable via `--json-out`)
and print per-frame summaries to stdout.
`README.md` next to the suite indexes the drivers and lists the
supporting modules.
Signed-off-by: Francis Williams <francis@fwilliams.info>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a
tests/benchmarks/tsdf_fusion/benchmark suite that compares fvdb against the public sparse-voxel libraries (nvblox, VDBFusion, Open3D) on real and synthetic RGB-D / LiDAR workloads.Pairs with the fvdb-core TSDF / ESDF / Occupancy / Decay PR (openvdb/fvdb-core#656, itself stacked on #655). The drivers exercise the new
Grid.integrate_tsdf_frames,integrate_tsdf_from_points,integrate_occupancy_from_points,compute_esdf, anddecay_and_pruneAPIs.Per-dataset bench drivers (
tests/benchmarks/tsdf_fusion/cross_library/)bench_open3d_vs_fvdb.pybench_replica_full.pybench_esdf_replica.pybench_kitti.pybench_esdf_kitti.pybench_occupancy_kitti.pybench_decay_kitti.pybench_mai_city.pybench_seven_scenes.pybench_esdf_vs_nvblox.pybench_occupancy_vs_nvblox.pySupporting modules
kitti_loader.py,replica_loader.py,mai_city_loader.py,seven_scenes_loader.py— minimal per-dataset loaders that yield(intrinsics, cam_to_world, depth_or_points)tuples.download_kitti.py,download_replica.py,download_replica_zip.py— resumable, parallel-stream downloaders for the public dataset archives.nvblox_runner.py— Python wrapper around the nvblox CLI that matches the cross-library interface used by the bench drivers above.install_nvblox.sh,install_vdbfusion.sh— reproducible install recipes for the two C++ comparison libraries (each builds against the fvdb conda env's TBB / Blosc / Boost).data/README.md,data/.gitignore— conventions for placing dataset trees underdata/. The dataset files themselves are not vendored.Each driver accepts
--helpfor its full argument surface and writes a JSON results file alongside per-frame stdout summaries.Test plan
A no-external-dataset smoke test runs in a few seconds and is the easiest way to verify the setup:
The dataset-bound drivers expect the public archives under
tests/benchmarks/tsdf_fusion/data/<dataset>/(gitignored; downloaders provided in the same directory).bench_replica_full.pyagainst a Replica scene).install_nvblox.sh/install_vdbfusion.shproduce working installs from a clean conda env (these are scripts, not part of the fvdb wheel).