Skip to content

README.md

Latest commit

 

History

History
119 lines (82 loc) · 4.42 KB

README.md

File metadata and controls

119 lines (82 loc) · 4.42 KB

3D Face Reconstruction with DECA

This repository contains a working setup of DECA (Detailed Expression Capture and Animation) for single-image 3D face reconstruction and expression transfer. It includes:

  • Core DECA library and demo scripts (DECA/)
  • Sample inputs and expected outputs (DECA/test_samples/)
  • An optional Flask-based web UI to run reconstruction from the browser (mesh_viewer_web/)

The demos reconstruct a textured 3D face mesh from an input image and optionally transfer expressions from a second image.

Requirements

  • Python 3.7
  • PyTorch 1.6.0 and torchvision 0.7.0 (as pinned in DECA/requirements.txt)
  • Optional: PyTorch3D if you choose the pytorch3d rasterizer; the demos can run with the built-in “standard” rasterizer to avoid PyTorch3D

Notes

  • Installing PyTorch3D depends on a matching CUDA/PyTorch toolchain. If you prefer a simpler setup or CPU-only, use --rasterizer_type standard in the demo scripts.
  • The face detector uses face-alignment (installed via requirements).

Python packages

All required Python packages are pinned in DECA/requirements.txt and include:

  • numpy==1.18.5, scipy==1.4.1, scikit-image==0.15, opencv-python>=4.1.1, PyYAML
  • torch==1.6.0, torchvision==0.7.0
  • face-alignment, yacs==0.1.8, kornia==0.4.0, ninja, fvcore
  • PyTorch3D is optional and not pinned here (install only if you choose --rasterizer_type pytorch3d)

Setup

You can set up the environment with conda or a virtualenv. Run these commands from the DECA/ folder so relative paths resolve correctly.

Option A — conda (creates an environment named deca-env)

cd DECA
bash install_conda.sh

Option B — virtualenv (Python 3.7)

### 1) Single-image reconstruction

Runs on images in `DECA/test_samples/examples` and saves outputs per image.

Standard rasterizer (simpler, no PyTorch3D required):

```bash
python DECA/demos/demo_reconstruct.py \
	--inputpath DECA/test_samples/examples \
	--savefolder DECA/test_samples/examples/results \
	--rasterizer_type standard \
	--saveDepth true \
	--saveImages true

Notes

  • Set --device cpu to force CPU; omit or use --device cuda for GPU.
  • If you have all texture assets and want the FLAME texture model, set --useTex true.
  • --render_orig true renders in the original image size and is supported with --rasterizer_type standard.

Outputs

  • A results folder per input image containing visualizations and optional files such as:
    • <name>_depth.jpg, <name>_kpt2d.txt, <name>_kpt3d.txt
    • <name>.obj and <name>_detail.obj (if --saveObj true)
    • <name>.mat (if --saveMat true)

2) Expression transfer

Transfers the expression from --exp_path to the identity in --image_path.

python DECA/demos/demo_transfer.py \
	--image_path DECA/test_samples/uniquify_data/tungt_20240909_0202.jpg \
	--exp_path DECA/test_samples/exp/6.jpg \
	--savefolder DECA/test_samples/animation_results \
	--rasterizer_type standard \
	--saveAnimation true \
	--saveObj true \
	--saveImages true

This produces an animation visualization image and, if requested, per-case assets under the save folder.

Optional: Web viewer

The mesh_viewer_web/ folder provides a minimal Flask app to run reconstruction from the browser and preview outputs.

  1. Install Flask dependencies in your environment:
pip install flask werkzeug
  1. Start the app:
python mesh_viewer_web/app.py
  1. Open http://127.0.0.1:5000 in your browser and upload an image.

Notes

  • The web app runs the reconstruction with --rasterizer_type pytorch3d by default. Install PyTorch3D or change that flag in mesh_viewer_web/app.py to standard if you want to avoid PyTorch3D.
  • Outputs are written under mesh_viewer_web/static/.

Troubleshooting

  • Missing assets: Ensure all files listed in the Model assets section exist under DECA/data/ or update their paths in DECA/decalib/utils/config.py.
  • PyTorch3D install errors: Prefer the “standard” rasterizer to skip PyTorch3D, or install a PyTorch3D build matching your PyTorch/CUDA versions.
  • CPU vs GPU: Use --device cpu to run without CUDA. GPU acceleration requires a compatible CUDA toolkit and driver.
  • Face detection/cropping: You can switch detectors via --detector (see DECA/decalib/datasets/detectors.py) and toggle cropping via --iscrop.

License

Please see DECA/LICENSE for licensing terms. Some assets (e.g., FLAME model files) may require separate licenses and are not redistributed here.