Skip to content

feat(nuscenes): add nuScenes to NCore V4 converter#128

Open
janickm wants to merge 1 commit into
NVIDIA:mainfrom
janickm:issue-123-jammy-quokka
Open

feat(nuscenes): add nuScenes to NCore V4 converter#128
janickm wants to merge 1 commit into
NVIDIA:mainfrom
janickm:issue-123-jammy-quokka

Conversation

@janickm
Copy link
Copy Markdown
Collaborator

@janickm janickm commented May 21, 2026

Summary

Adds a nuScenes dataset converter to NCore V4 format, following the existing KITTI converter pattern.

image

Closes #123

Features

  • Supports all nuScenes versions (v1.0-mini, v1.0-trainval, v1.0-test)
  • 6 cameras with intrinsics, extrinsics, and frame data (global shutter, undistorted)
  • LIDAR_TOP with unstructured point clouds (direction + distance)
  • Per-point timestamps estimated from azimuth (Velodyne HDL-32E, ~50ms scan duration)
  • 3D cuboid annotations in world frame from keyframe sample_annotations
  • Scene selection via --scene-token or --scene-name
  • Outputs itar or directory store formats
  • Integration tests (22 tests pass with v1.0-mini)

Usage

bazel run //tools/data_converter/nuscenes -- \
    --root-dir /path/to/nuscenes \
    --output-dir /path/to/output \
    nuscenes \
    --version v1.0-mini \
    --scene-name scene-0061

Design Decisions

  • Lidar: Unstructured (model_element=None). Structured HDL-32E model deferred to follow-up.
  • Cuboids: Stored in world frame (reference_frame_id="world"). The V4 reader's transform() handles re-projection to sensor frames at runtime.
  • Per-point timestamps: Estimated from azimuth angle (same approach as KITTI). nuScenes doesn't provide per-point timing.
  • Camera shutter: ShutterType.GLOBAL -- nuScenes provides single capture timestamp with no RS metadata.
  • No dynamic flag: V4 format doesn't use per-point dynamic flags (was V3 only).

Test Instructions

NUSCENES_DIR=/path/to/nuscenes NUSCENES_VERSION=v1.0-mini \
    bazel test //tools/data_converter/nuscenes:pytest_converter_3_11 --test_output=all \
    --test_env=NUSCENES_DIR=/path/to/nuscenes --test_env=NUSCENES_VERSION=v1.0-mini

Files

  • deps/pip/requirements_nuscenes.in -- new pip deps (nuscenes-devkit, pyquaternion)
  • deps/pip/BUILD.bazel -- include new requirements in pip_compile
  • deps/pip/requirements_3_11.in / .txt -- updated lockfile
  • tools/data_converter/nuscenes/ -- converter, utils, CLI, BUILD, test, README, NOTICE

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 21, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@janickm janickm self-assigned this May 21, 2026
@janickm janickm force-pushed the issue-123-jammy-quokka branch 19 times, most recently from f05c1db to f8deb39 Compare May 29, 2026 12:05
@janickm janickm force-pushed the issue-123-jammy-quokka branch 8 times, most recently from 2d148d4 to 988e4da Compare May 29, 2026 14:36
Implements a converter from nuScenes dataset format to NCore V4 component-based
format, following the KITTI converter pattern.

Features:
- Supports all nuScenes versions (v1.0-mini, v1.0-trainval, v1.0-test)
- 6 cameras with intrinsics, extrinsics, and frame data
- LIDAR_TOP with unstructured point clouds (direction + distance)
- Per-point timestamps estimated from azimuth (HDL-32E, ~50ms scan)
- 3D cuboid annotations in world frame from keyframe sample_annotations
- Scene selection via --scene-token or --scene-name
- Outputs itar or directory store formats

Sensor assumptions documented:
- Cameras: global shutter, undistorted (zero distortion coeffs)
- Lidar: Velodyne HDL-32E at 20Hz, unstructured (model_element=None)
- Cuboids: world frame, keyframes only

Tested with v1.0-mini (22 tests pass, both itar and directory formats).

Closes: NVIDIA#123
@janickm janickm force-pushed the issue-123-jammy-quokka branch from 988e4da to 9d2d19c Compare May 29, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nuScenes and Argoverse 2 support

1 participant