What Does the 1×5 Input Dimension Represent in a LiDAR Backbone Model? BEVFusion

[dheepak-gan]

Hi, I am seeking help with an issue I’m currently facing. For the LiDAR backbone model (ONNX), the required input dimension is 1×5. Since LiDAR data is usually in an unstructured point cloud format, we need to voxelize it before passing it to the CNN sparse convolution layer, so that the structured data can be processed properly.

Typically, the input to a sparse convolution layer is features + coordinates. However, in this LiDAR backbone model, the input is expected to be 1×5, and I am unsure what these dimensions represent.

Does 1×5 mean one voxel with five features: (batch index, x index, y index, z index, and mean intensity of points inside the voxel)?

Or does it mean one voxel with: (x index, y index, z index, mean intensity, and number of points per voxel)? or other features different from these two?

I am stuck at this point and would like clarification on what the 1×5 input exactly represents. Also, does this mean the LiDAR backbone model only processes a single voxel at a time, or is this just an example format of how the model execution is structured?

Any guidance or explanation would be greatly appreciated.
Thanks!

[uff1199]

I would say the 1 x 5 represents the num_features in the PointCloud Voxel, hence the X, Y, Z, Intensity and Reflectivity values obtained by the LiDAR sensor and aggregated in one voxel. Also I face problems when running it with only 1 x 4 e. g. by skipping the reflectifity values.

[dheepak-gan]

If I have 20,000 points, can we modify it to have a shape of 20,000×5?
Also, can we reshape a tensor with spatial dimensions of 1440×1440×41 into 1×5×1440×1440×41 and process it, or is that approach incorrect?
And in the input tensor(1x5x1440x1440x41), can we include both active and non-active voxel features? and proceed as mentioned above?