LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control

Jianzhu Guo ^1†  Dingyun Zhang ^1,2  Xiaoqiang Liu ¹  Zhizhou Zhong ^1,3  Yuan Zhang ¹ 

Pengfei Wan ¹  Di Zhang ¹ 

¹ Kuaishou Technology  ² University of Science and Technology of China  ³ Fudan University 

🔥 For more results, visit our homepage 🔥

🔥 Updates

• 2024/07/17: 🍎 We support macOS with Apple Silicon, modified from jeethu's PR #143.
• 2024/07/10: 💪 We support audio and video concatenating, driving video auto-cropping, and template making to protect privacy. More to see here.
• 2024/07/09: 🤗 We released the HuggingFace Space, thanks to the HF team and Gradio!
• 2024/07/04: 😊 We released the initial version of the inference code and models. Continuous updates, stay tuned!
• 2024/07/04: 🔥 We released the homepage and technical report on arXiv.

Introduction

This repo, named LivePortrait, contains the official PyTorch implementation of our paper LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control. We are actively updating and improving this repository. If you find any bugs or have suggestions, welcome to raise issues or submit pull requests (PR) 💖.

More interesting results can be found in our Homepage 😊

Below are the results of inferring one frame on an RTX 4090 GPU using the native PyTorch framework with torch.compile:

Model	Parameters(M)	Model Size(MB)	Inference(ms)
Appearance Feature Extractor	0.84	3.3	0.82
Motion Extractor	28.12	108	0.84
Spade Generator	55.37	212	7.59
Warping Module	45.53	174	5.21
Stitching and Retargeting Modules	0.23	2.3	0.31

Note: the listed values of Stitching and Retargeting Modules represent the combined parameter counts and the total sequential inference time of three MLP networks.

Acknowledgements

We would like to thank the contributors of FOMM, Open Facevid2vid, SPADE, InsightFace repositories, for their open research and contributions.

Citation 💖

If you find LivePortrait useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:

@article{guo2024live,
  title   = {LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control},
  author  = {Jianzhu Guo and Dingyun Zhang and Xiaoqiang Liu and Zhizhou Zhong and Yuan Zhang and Pengfei Wan and Di Zhang},
  year    = {2024},
  journal = {arXiv preprint:2407.03168},
}