Query on Wi-Fi 3D Pose Testing Implementation about testing process

[werringwu]

Experiencing an AttributeError during testing with PETR WiFi model:
and config in petr_wifi.py the meta_keys is empty, and the image_meta always be empty. And I don't know how to deal with it. Could you help me?

[silentchaostheory]

Hello,

I'm also having fun with testing my trained model. The absence of these keys is an issue caused by the --show and --show_dir parameters which trigger the calculation of grounded truths. If you just use --out and --eval mpjpe, it should use existing datatypes from the pipeline.

I also found examples of possible meta keys for the pipeline's Collect step:

meta_keys=( # ALL meta keys needed for inference/evaluation
                    'filename', 
                    'ori_filename', 
                    'ori_shape', 
                    'img_shape',
                    'pad_shape', 
                    'scale_factor', 
                    'flip', 
                    'flip_direction',
                    'img_norm_cfg',
                    # *** CRITICAL: ADD KEYS NEEDED FOR EVALUATION HERE ***
                    # The evaluation metrics often need the ground truth data keys
                    'gt_keypoints', 
                    'gt_labels',
                )

The error I received was for the img_nrom_cfg key but the gt_keypoints and gt_labels were suggested as well.

[silentchaostheory]

Update:
The Transforms pipeline for test_pipeline, data[val], and data[test] is missing a step that adds the img_norm_cfg to the results based on what I was able to find. I do not think the test or val pipelines from the petr_wifi.py config were ever used for this project without the train step running first.

First, in the opera.datasets.pipelines I created a new pipeline, add_meta_info.py, to add a dummy results value:

# opera/datasets/pipelines/add_meta_info.py
from ..builder import PIPELINES   # matches other pipeline modules in this repo

@PIPELINES.register_module()
class AddMetaInfo:
    """Insert a map of metadata entries into results if missing.

    Usage in config: dict(type='opera.AddMetaInfo', info=dict(img_norm_cfg=...))
    """
    def __init__(self, results=None):
        self.results = results or {}

    def __call__(self, results):
        results.setdefault(
            'img_norm_cfg',
             dict(mean=[0, 0, 0], std=[1, 1, 1], to_rgb=False)
        )

I then added the new pipeline to the init.py:

from .add_meta_info import AddMetaInfo
from .formatting import DefaultFormatBundle
from .loading import LoadAnnotations
from .transforms import Resize, RandomFlip, RandomCrop, KeypointRandomAffine

__all__ = [
    'AddMetaInfo', 'DefaultFormatBundle', 'LoadAnnotations',
    'Resize', 'RandomFlip', 'RandomCrop', 'KeypointRandomAffine'
]

And then ran the import to with the -c option so the cython build would update the pycache.
python -c "import opera.datasets.pipelines.add_meta_info; print('Loaded:', opera.datasets.pipelines.add_meta_info)"

I added this at the top of the config:
img_norm_cfg = dict(mean=[0, 0, 0], std=[1, 1, 1], to_rgb=False)
as first step in transforms array for val and test:

            dict(
                type='opera.AddMetaInfo',
                meta=dict(
                    img_norm_cfg=dict(mean=[0, 0, 0], std=[1, 1, 1], to_rgb=False)
                )
            ),

As well as adding the img_norm_cfg to the meta_key for the mmdet.Collect step of the pipeline.

Then, add this to tools/test.py if you get an error for mmcv not having the new opera pipeline registered.

from opera.datasets.pipelines.add_meta_info import AddMetaInfo
from opera.datasets.builder import PIPELINES as OPERA_PIPELINES
from mmdet.datasets.builder import PIPELINES as MMDET_PIPELINES

# Ensure both registries see it
OPERA_PIPELINES.register_module(module=AddMetaInfo)
if 'AddMetaInfo' not in MMDET_PIPELINES.module_dict:
    MMDET_PIPELINES.register_module(module=AddMetaInfo)

This did not work for me, but I am using Conda environment that I have frozen.

I am not trying a modified pipeline for testing that removes the type='mmdet.MultiScaleFlipAug' wrapper for the other steps as there isn't actually a transform occurring.

[werringwu]

Hello, is it already possible for you to proceed? @silentchaostheory

[silentchaostheory]

It removed the key error but now the torch.Tensor object is passed to the detectors forward_test function but it's expecting a list.

.../mmdet/models/detectors/base.py", line 124, in forward_test
    raise TypeError(f'{name} must be a list, but got {type(var)}')
TypeError: imgs must be a list, but got <class 'torch.Tensor'>

Is that better? Not really sure.

[silentchaostheory]

Update:
I tried wrapping the tensor assuming it was an issue of the single sample being passed as an object instead of a list and a later check failed for the dimensionality of the data object because of this wrapping.

Traceback (most recent call last):
  File "/scratch/users/aad10/1506143-CKOA/Person-in-WiFi-3D-repo/tools/test.py", line 286, in <module>
    main()
  File "/scratch/users/aad10/1506143-CKOA/Person-in-WiFi-3D-repo/tools/test.py", line 244, in main
    outputs = single_gpu_test(model, data_loader, args.show, args.show_dir,
  File "/hpchome/aad10/project/Person-in-WiFi-3D-repo/opera/apis/test.py", line 42, in single_gpu_test
    imgs = tensor2imgs(img_tensor, **img_metas[0]['img_norm_cfg'])
  File "/hpchome/aad10/.conda/envs/wifipose3d/lib/python3.9/site-packages/mmcv/image/misc.py", line 34, in tensor2imgs
    assert torch.is_tensor(tensor) and tensor.ndim == 4
AssertionError

It would be helpful for someone who understands the reason these assertions and type checks were implemented to provide input on these errors. It seems like someone assumed the val and test pipelines would work independent of the train pipeline without performing any verification of that. The PETR examples I can find suggests using the dist_test.sh script in the /tools directory of the project. This does nothing unique compared to the script I am using except that it passes pytorch as the launcher.

[silentchaostheory]

Update 2:
Discovered what is likely the important difference between the tool/train.py and tool/test.py and it's super annoying that it's just sitting there without any validation messages considering how many exceptions are included in the test main. This is the likely cause of the seemingly contradictory data.test.pipeline config and the observed behavior, cfg = compat_cfg(cfg). This is imported from mmdet.utils and is likely the previous iteration of the current releases compat_config. Inside of this util, the test_pipeline parameter suddenly becomes your actual config, regardless of what was previously in the data.test config. If the concern was compatibility, there should be a comparison between the new and old pipeline configuration. Going to try moving my pipeline to the test_pipeline parameter and then setting the data.val and data.test pipeline object equal to test_pipeline.

[Alexbao1]

I encountered the same issue during model training. Following Copilot's suggestion, I changed all meta_keys=[] to meta_keys=['filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'img_norm_cfg']) in petr_wifi.py, and manually added the meta_key content in get_item_single_frame in opera/datasets/wifi_pose.py:

result = dict(img=csi, gt_keypoints=keypoint, gt_labels=gt_labels, gt_bboxes=gt_bboxes, gt_areas=gt_areas, img_name=data_name)
# DEBUG Ensure mmdet Collect can build non-empty img_metas for test/val
result['filename'] = result.get('img_name', str(data_name))
h = csi.shape[-2] if hasattr(csi, 'shape') and len(csi.shape) >= 2 else 0
w = csi.shape[-1] if hasattr(csi, 'shape') and len(csi.shape) >= 2 else 0
result['ori_shape'] = (h, w, 1)
result['img_shape'] = (h, w, 1)
result['pad_shape'] = (h, w, 1)
result['scale_factor'] = 1.0
result['flip'] = False
result['img_norm_cfg'] = dict(mean=[0, 0, 0], std=[1, 1, 1], to_rgb=False)

Finally, I commented out img_meta = img_meta.data[0] in forward_test of /third_party/mmdet/mmdet/models/detectors/base.py.
Although I'm not sure if this approach has any potential risks, at least it allows normal training to proceed now.

[silentchaostheory]

I encountered the same issue during model training. Following Copilot's suggestion, I changed all meta_keys=[] to meta_keys=['filename', 'ori_shape', 'img_shape', 'pad_shape', 'scale_factor', 'flip', 'img_norm_cfg']) in petr_wifi.py, and manually added the meta_key content in get_item_single_frame in opera/datasets/wifi_pose.py:

result = dict(img=csi, gt_keypoints=keypoint, gt_labels=gt_labels, gt_bboxes=gt_bboxes, gt_areas=gt_areas, img_name=data_name)

DEBUG Ensure mmdet Collect can build non-empty img_metas for test/val

result['filename'] = result.get('img_name', str(data_name))
h = csi.shape[-2] if hasattr(csi, 'shape') and len(csi.shape) >= 2 else 0
w = csi.shape[-1] if hasattr(csi, 'shape') and len(csi.shape) >= 2 else 0
result['ori_shape'] = (h, w, 1)
result['img_shape'] = (h, w, 1)
result['pad_shape'] = (h, w, 1)
result['scale_factor'] = 1.0
result['flip'] = False
result['img_norm_cfg'] = dict(mean=[0, 0, 0], std=[1, 1, 1], to_rgb=False)
Finally, I commented out img_meta = img_meta.data[0] in forward_test of /third_party/mmdet/mmdet/models/detectors/base.py. Although I'm not sure if this approach has any potential risks, at least it allows normal training to proceed now.

Yes, I figured the pipeline's missing step if you don't want to comment that out.

test_pipeline = [
    dict(
        type='opera.DefaultFormatBundle',
        extra_keys=['gt_keypoints', 'gt_labels']),
    dict(
        type='mmdet.Collect', 
        keys=['img', 'gt_bboxes', 'gt_labels', 'gt_keypoints', 'gt_areas'], 
        meta_keys=[
            'img_norm_cfg', # The config used for normalization
            'ori_shape',    # Original shape of the input
            'img_shape',    # Current shape after resizing/padding
            'scale_factor', # Scale used for resizing
            'flip'          # Flip status (often needed for TTA)
        ],
    ),
    dict(type='WrapFieldsToLists')
]

The mmdet.WrapFieldsToList pipeline should wrap the values created by the mmdet.Collect built keys so the forward_test is happy finally. I'm trying it now, will update. Glad you found that to be the same issue. It's just like peeling an onion but expecting a specific number of layers.

[silentchaostheory]

I updated the opera/datasets/wifi_pose.py to included the meta_keys in the results object it builds. The mmdet.Collect then takes the keys listed in the meta_keys and puts them in the img_metas dict and then wrap that as a list. The mmdet.WrapFieldsToList then adds the list layer batching, though we don't need that for test. I tired building img_metas in wifi_pose.py but they get overwritten in the Collect step.

    def __init__(self, dataset_root, pipeline, mode, **kwargs):
        self.img_norm_cfg = kwargs.get('img_norm_cfg', DEFAULT_IMG_NORM_CFG)
        self.img_file_type = kwargs.get('img_file_type', 'jpg')
        self.scale_factor = kwargs.get('scale_factor', np.array(1.0, dtype=np.float32))
        self.flip = kwargs.get('flip', False)
        self.data_root = dataset_root
        self.pipeline = Compose(pipeline)
        self.filename_list = self.load_file_name_list(os.path.join(self.data_root, mode + '_data_list.txt'))
        self._set_group_flag()
...
        img_meta = {
            'filename': csi_path,
            'img_norm_cfg': self.img_norm_cfg,
            'ori_shape': ori_shape,
            'img_shape': csi.shape[::-1],
            'scale_factor': self.scale_factor,
            'flip': self.flip,
            'ori_filename': data_name,
            'img_file_type': self.img_file_type,
        }

        result = dict(img=csi, gt_keypoints=keypoint, gt_labels = gt_labels, gt_bboxes = gt_bboxes, gt_areas = gt_areas, img_name = data_name)
        result = result | img_meta
        return result

You can use the --show and --show-dir if you update the opera/apis/test.py to build the file name and type string for saving. I'm not sure why there isn't an existing string for that, maybe the filename meta-key is actually meant to satisfy that requirement, at which point the wifi_pose.py should be updated to take a file type supported by mmcv.image.tensor2imgs.

        if show or out_dir:

            if batch_size == 1 and isinstance(data['img'][0], torch.Tensor):
                img_tensor = data['img'][0]
            else:
                img_tensor = data['img'][0].data[0]
            img_metas = data['img_metas'][0].data[0]
            
            # we have to update the img and img_shape for this project
            img_tensor = img_tensor[:, 0, :, :, :]
            W, H, C, _ = img_tensor.shape 
            img_metas[0]['img_shape'] = (H, W, C)
            img_metas[0]['ori_shape'] = (H, W, C)

            imgs = tensor2imgs(img_tensor, **img_metas[0]['img_norm_cfg'])

            assert len(imgs) == len(img_metas)

            for i, (img, img_meta) in enumerate(zip(imgs, img_metas)):
                h, w, _ = img_meta['img_shape']
                img_show = img[:h, :w, :]

                ori_h, ori_w = img_meta['ori_shape'][:-1]
                img_show = mmcv.imresize(img_show, (ori_w, ori_h))

                if out_dir:
                    out_file = osp.join(out_dir, img_meta['ori_filename']+'.'+img_meta['img_file_type'])
                else:
                    out_file = None

I'm not sure how meaningful the images are in this context as they input was not an image as CV libraries usually expect. For this reason, I had to change the dimensions of the img to remove the receiver. It might be useful to rotate the result or each antenna group based on its location in relation to the camera but I would assume that has already occurred in pre-processing.

[wzx-20021022]

@silentchaostheory 你好请问现在问题解决了吗，我遇到的问题也是img_nrom_cfg

[silentchaostheory]

I was able to proceed. There's an example of a working configuration file here: https://doi.org/10.5281/zenodo.20241335

…

On Tue, Aug 4, 2026, 2:32 AM wzx-20021022 ***@***.***> wrote: *wzx-20021022* left a comment (aiotgroup/Person-in-WiFi-3D-repo#14) <#14 (comment)> @silentchaostheory <https://github.com/silentchaostheory> 你好请问现在问题解决了吗，我遇到的问题也是img_nrom_cfg — Reply to this email directly, view it on GitHub <#14?email_source=notifications&email_token=ANSCO3OQUCDZOQ7NJJEXKK35IGGRDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMJXGU4TIMBZGY42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-5175940969>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSCO3NTO3I66SHC772TBVT5IGGRDAVCNFSNUABFKJSXA33TNF2G64TZHM3TMNRXGEZTQNRXHNEXG43VMU5TGMJWGU4DOMJYG42KC5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/ANSCO3OVWRFZKERMRBLMWI35IGGRDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMJXGU4TIMBZGY42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJKTGN5XXIZLSL5UW64Y> and Android <https://github.com/notifications/mobile/android/ANSCO3L2RTLS6BZU2KYBNPD5IGGRDA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTKMJXGU4TIMBZGY42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLTGN5XXIZLSL5QW4ZDSN5UWI>. Download it today! You are receiving this because you were mentioned.Message ID: ***@***.***>