PISCO: Precise Video Instance Insertion with Sparse Control

This repo hosts the official implementation of PISCO: Precise Video Instance Insertion with Sparse Control

Video Demos

Instance Insertion

Before
After

Creative

Before
After

Reposition

Before
After

Resize

Before
After

Simulation

Before
After

TODO list

• Release Inference Code
• Release Development Tools
• Release Training Code
• Release Training Set

Installation

conda create -n pisco python=3.12
conda activate pisco
# Install the correct version of torch
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
# Install deepspeed if training 14B model
pip install deepspeed

Inference

# 1.3B 480p
python inference/pretrained/infer_1.3B.py
# 1.3B 720p
python inference/pretrained/infer_1.3B_720p.py
# 14B 480p
python inference/pretrained/infer_14B.py
# 14B 720p
python inference/pretrained/infer_14B_720p.py

Acknowledgments

This repo is built upon the Diffsynth-Studio codebase. Thanks to the authors for their great work!