ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization

Wenhao Shen  Wanqi Yin  Xiaofeng Yang  Cheng Chen  Chaoyue Song  Zhongang Cai  Lei Yang  Hao Wang  Guosheng Lin 

ICML 2025

---

---

🛠️ Install

Set up the environment

conda create -n adhmr python=3.8 -y
conda activate adhmr
conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 cudatoolkit=11.3 -c pytorch -y
pip install mmcv-full==1.7.1 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.12.0/index.html
pip install -r requirements.txt

# install pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git"

# install mmpose
cd HMR-Scorer/main/transformer_utils
pip install -v -e .
cd -

HMR-Scorer

• cd HMR-Scorer/
• download all datasets
  • 3DPW
  • Human3.6M
  • HI4D
  • BEDLAM
  • RenBody (DNA-Rendering)
  • GTA-Human
  • SPEC
  • InstaVariety
• process all datasets into HumanData format, except the following:
  • Human3.6M.
  • follow OSX in preparing this dataset.
• follow OSX in preparing pretrained ViTPose models. Download the ViTPose pretrained weights for ViT-small and ViT-huge from here.
• download SMPL-X and SMPL body models.
• download HMR-Scorer test datasets from OneDrive (DNA-Rendering and GTA-Human) and put them under dataset/cache_scorer_eval.

The file structure should be like:

HMR-Scorer/
├── common/
│   └── utils/
│       └── human_model_files/  # body model
│           ├── smpl/
│           │   ├──SMPL_NEUTRAL.pkl
│           │   ├──SMPL_MALE.pkl
│           │   └──SMPL_FEMALE.pkl
│           └── smplx/
│               ├──MANO_SMPLX_vertex_ids.pkl
│               ├──SMPL-X__FLAME_vertex_ids.npy
│               ├──SMPLX_NEUTRAL.pkl
│               ├──SMPLX_to_J14.pkl
│               ├──SMPLX_NEUTRAL.npz
│               ├──SMPLX_MALE.npz
│               └──SMPLX_FEMALE.npz
├── data/
├── main/
├── output/
├── pretrained_models/  # pretrained ViT-Pose, SMPLer_X and mmdet models
│   ├── smpler_x_s32.pth.tar
│   ├── smpler_x_b32.pth.tar
│   ├── smpler_x_l32.pth.tar
│   ├── smpler_x_h32.pth.tar
│   ├── vitpose_small.pth
│   ├── vitpose_base.pth
│   ├── vitpose_large.pth
│   └── vitpose_huge.pth
└── dataset/  
    ├── 3DPW/       
    ├── Human36M/             
    ├── HI4D
    ├── BEDLAM/      
    ├── RenBody/      
    ├── GTA_Human2/           
    ├── CHI3D/       
    ├── InstaVariety/         
    ├── SPEC/   
    ├── cache_scorer_eval/      # HMR-Scorer test datasets
    └── preprocessed_datasets/  # HumanData files

ADHMR

• cd ADHMR/
• follow ScoreHypo to prepare data.

🚀 Pretrained Models

• download ckeckpoint of HMR-Scorer from OneDrive
  • put it under HMR-Scorer/output/
• download checkpoint of ADHMR from OneDrive
  • put them under ADHMR/experiment/hyponet

📝 Evaluation

HMR-Scorer

# To eval the model HMR-Scorer/output/{TRAIN_OUTPUT_DIR}
export PYTHONPATH=$PYTHONPATH:/path/to/ADHMR/HMR-Scorer
cd ./HMR-Scorer/main/
JOB_NAME=GTA_Human2 / RenBody_HiRes
torchrun test_scorer.py --num_gpus 1 --exp_name output/scorer_test_${JOB_NAME} --result_path train_scorer_b5_2d_1118_all_loss_20241120_144943 --ckpt_idx 20 --testset ${JOB_NAME}

ADHMR

# To eval on 3DPW
torchrun --nproc_per_node=2 --master_port=23452 main/main.py --config config/test/test-3dpw-custom.yaml --exp experiment/scorenet --doc 3dpw --validate --multihypo_n 100 --batch_size 80
# To eval on Human3.6M
torchrun --nproc_per_node=2 --master_port=23452 main/main.py --config config/test/test-h36m-custom.yaml --exp experiment/scorenet --doc h36m --validate --multihypo_n 100 --batch_size 80

📚 Citation

If you find our work useful for your research, please consider citing the paper:

@inproceedings{shen2025adhmr,
  title={ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization},
  author={Shen, Wenhao and Yin, Wanqi and Yang, Xiaofeng and Chen, Cheng and Song, Chaoyue and Cai, Zhongang and Yang, Lei and Wang, Hao and Lin, Guosheng},
  booktitle={International Conference on Machine Learning},
  year={2025},
  organization={PMLR}
}

👏 Acknowledgement

This repo is built on the excellent work SMPLer-X, ScoreHypo. Thanks for their great projects.

Explore More SMPLCap Projects

• [arXiv'25] SMPLest-X: An extended version of SMPLer-X with stronger foundation models.
• [ECCV'24] WHAC: World-grounded human pose and camera estimation from monocular videos.
• [CVPR'24] AiOS: An all-in-one-stage pipeline combining detection and 3D human reconstruction.
• [NeurIPS'23] SMPLer-X: Scaling up EHPS towards a family of generalist foundation models.
• [NeurIPS'23] RoboSMPLX: A framework to enhance the robustness of whole-body pose and shape estimation.
• [ICCV'23] Zolly: 3D human mesh reconstruction from perspective-distorted images.
• [arXiv'23] PointHPS: 3D HPS from point clouds captured in real-world settings.
• [NeurIPS'22] HMR-Benchmarks: A comprehensive benchmark of HPS datasets, backbones, and training strategies.