MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space

Lixing Xiao¹ · Shunlin Lu ² · Huaijin Pi³ · Ke Fan⁴ · Liang Pan³ · Yueer Zhou¹ · Ziyong Feng⁵ ·
Xiaowei Zhou¹ · Sida Peng^1† · Jingbo Wang<sup>6

1</sup>Zhejiang University ²The Chinese University of Hong Kong, Shenzhen ³The University of Hong Kong
⁴Shanghai Jiao Tong University ⁵DeepGlint ⁶Shanghai AI Lab
Arxiv 2025

TODO List

• Release the processing script of 272-dim motion representation.
• Release the processed 272-dim Motion Representation of HumanML3D dataset. Only for academic usage.
• Release the training code and checkpoint of our TMR-based motion evaluator trained on the processed 272-dim HumanML3D dataset.
• Release complete code for MotionStreamer.

🏃 Motion Representation

For more details of how to obtain the 272-dim motion representation, as well as other useful tools (e.g., Visualization and Conversion to BVH format), please refer to our GitHub repo.

Installation

🐍 Python Virtual Environment

conda env create -f environment.yaml
conda activate mgpt

🤗 Hugging Face Mirror

Since all of our models and data are available on Hugging Face, if Hugging Face is not directly accessible, you can use the HF-mirror tools following:

pip install -U huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com

📥 Data Preparation

To facilitate researchers, we provide the processed 272-dim Motion Representation of HumanML3D dataset on Hugging Face.

❗️❗️❗️ The processed data is solely for academic purposes. Make sure you read through the AMASS License. Download the processed 272-dim HumanML3D dataset following:

huggingface-cli download --repo-type dataset --resume-download lxxiao/272-dim-HumanML3D --local-dir ./humanml3d_272
cd ./humanml3d_272
unzip texts.zip
unzip motion_data.zip

The dataset is organized as:

./humanml3d_272
  ├── mean_std
      ├── Mean.npy
      ├── Std.npy
  ├── split
      ├── train.txt
      ├── val.txt
      ├── test.txt
  ├── texts
      ├── 000000.txt
      ...
  ├── motion_data
      ├── 000000.npy
      ...

🚀 Training

1. Training our TMR-based motion evaluator on the processed 272-dim HumanML3D dataset following:

   bash TRAIN_evaluator_272.sh

   After training for 100 epochs, the checkpoint will be stored at: Evaluator_272/experiments/temos/EXP1/checkpoints/.

   We provide the evaluator checkpoint on Hugging Face, download it following:

   python humanml3d_272/prepare/download_evaluator_ckpt.py

   The downloaded checkpoint will be stored at: Evaluator_272/.

📍 Evaluation

1. Evaluate the metrics of the processed 272-dim HumanML3D dataset following:

   bash EVAL_GT.sh

   ( FID, R@1, R@2, R@3, Diversity and MM-Dist (Matching Score) are reported. )

🌹 Acknowledgement

This repository builds upon the following awesome datasets and projects:

• AMASS
• HumanML3D
• T2M-GPT
• TMR
• OpenTMA

🤝🏼 Citation

If our project is helpful for your research, please consider citing :

@article{xiao2025motionstreamer,
      title={MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Space},
      author={Xiao, Lixing and Lu, Shunlin and Pi, Huaijin and Fan, Ke and Pan, Liang and Zhou, Yueer and Feng, Ziyong and Zhou, Xiaowei and Peng, Sida and Wang, Jingbo},
      journal={arXiv preprint arXiv:2503.15451},
      year={2025}
    }