Skip to content

How to generate DZYX_npy_f16 from the released npy_DZYX_complex radar tensors? #10

Description

@bbljy

Dear authors,

Thank you for releasing the RT-Pose dataset and code. I am currently
reproducing the HRRadarPose baseline using:

configs/cruw_pose/hr3d_one_hm_doppler.py

I found a difference between the released radar preprocessing output and
the radar data expected by the training code.

The released script data_processing/4Dradar2xyz.py saves the Cartesian
radar tensor to:

radar/npy_DZYX_complex

The saved tensor appears to be a complex-valued tensor with shape:

[64, 32, 128, 256] = [D, Z, Y, X]

However, the dataset loader used for training reads from:

DZYX_npy_f16

and directly converts the loaded array with:

arr_cube = np.load(...).astype(np.float32)

The main training configuration also uses:

RDR_TYPE = 'dzyx_real'
DZYX.NORMALIZING_VALUE = (0.0, 10.0)
backbone_cfg = 'hr_tiny_feat32_zyx_l4_in32'

Therefore, I would like to ask how the released 64-channel complex DZYX
tensor was converted into the DZYX_npy_f16 tensor used for training.

Could you please clarify the following details?

What is the exact shape and dtype of each DZYX_npy_f16 file?
How is the complex tensor converted to a real-valued tensor?
magnitude: abs(x)
power: abs(x) ** 2
real component
magnitude and phase
real and imaginary components
another representation
Was logarithmic or dB compression applied, such as:
log10(abs(x) + 1)
20 * log10(abs(x) + eps)
How were the 64 Doppler bins converted to the 32 input channels required
by hr_tiny_feat32_zyx_l4_in32?
selecting 32 bins
averaging adjacent bins
max pooling
positive/negative Doppler combination
another method
Was any background suppression, clipping, calibration, or dataset-level
normalization applied before saving the files?
Does NORMALIZING_VALUE=(0.0, 10.0) assume that the saved tensor has
already been transformed to approximately the range [0, 10]?
How is DZYX_npy_f16_complex generated for the phase/complex
configuration, and what tensor shape is expected by that configuration?

I have currently tested a custom preprocessing pipeline:

complex DZYX
-> magnitude
-> log10(x + 1)
-> retain all 64 Doppler channels
-> 64-channel HRNet3D

Using six sequences and training for 100 epochs, I obtained approximately:

MPJPE: 135.98 mm
ABS-MPJPE: 218.96 mm

However, I understand that this custom preprocessing may be different from
the preprocessing used in the paper.

Would it be possible to release the preprocessing script that generates
DZYX_npy_f16, or provide one corresponding pair of
npy_DZYX_complex and DZYX_npy_f16 sample files?

This missing conversion step appears to be important for reproducing the
reported result accurately.

Thank you very much for your help.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions