-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Description
Describe the bug
When using TiledCamera with "normals" enabled in data_types, the normals image obtained by camera.data.output["normals"] is incorrect. The incorrect image looks like vertical color stripes.
After comparing the implementation of TiledCamera (source/isaaclab/isaaclab/sensors/camera/tiled_camera.py) with that of Camera, I have located the issue: the normals annotator returns a databuffer of 4 channels (x, y, z, _), where the 4th channel is discarded by Camera._process_annotator_output to obtain the correct output of 3 channels (x, y, z). This process was not implemented in TiledCamera; instead, the 4-channel data buffer is directly reshaped into [B, H, W, C=3], leading to the incorrect output.
Steps to reproduce
To reproduce the bug, you can add "normals" to the data_types of the camera in source/isaaclab_tasks/isaaclab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py, and render one frame of the normals output.
In the environment config:
camera: TiledCameraCfg = TiledCameraCfg(
prim_path="{ENV_REGEX_NS}/Camera",
offset=TiledCameraCfg.OffsetCfg(pos=(-7.0, 0.0, 3.0), rot=(0.9945, 0.0, 0.1045, 0.0), convention="world"),
data_types=["rgb", "normals"],
spawn=sim_utils.PinholeCameraCfg(
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 20.0)
),
width=100,
height=100,
)In the rendering script:
# intialize and step the simulation once
normals_data = camera_data["normals"][0].cpu().numpy() # Shape: (H, W, 3)
# then visualize normals_data
System Info
- Commit: 44b9f1
- Isaac Sim Version: 5.1.0.0
- OS: Ubuntu 22.04.5 LTS
- GPU: NVIDIA GeForce RTX 4090
- CUDA: 12.6
- GPU Driver: 560.28.03
Additional context
I will submit a Pull Request containing a simple fix later.
Checklist
- I have checked that there is no similar issue in the repo (required)
- I have checked that the issue is not in running Isaac Sim itself and is related to the repo
Acceptance Criteria
Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.
- When
TiledCameraoutputs correct normals data by discarding the 4-th channel before reshaping the tensor.