Skip to content

Commit 82200ba

Browse files
yao531441pre-commit-ci[bot]
yao531441
and
pre-commit-ci[bot]
authored
Refine Readme for Animation. (#1873)
* Refine Readme for Animation. Signed-off-by: Yao, Qing <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Yao, Qing <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent e7c88ff commit 82200ba

File tree

2 files changed

+212
-133
lines changed

2 files changed

+212
-133
lines changed

‎comps/animation/deployment/docker_compose/README.md‎

Lines changed: 194 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
# Deploying Avatar Animation Service
2+
3+
This document provides a comprehensive guide to deploying the Avatar Animation microservice pipeline on Intel platforms.
4+
5+
This guide covers two deployment methods:
6+
7+
- [🚀 1. Quick Start with Docker Compose](#-1-quick-start-with-docker-compose): The recommended method for a fast and easy setup.
8+
- [🚀 2. Manual Step-by-Step Deployment (Advanced)](#-2-manual-step-by-step-deployment-advanced): For users who want to build and run each container individually.
9+
10+
## 🚀 1. Quick Start with Docker Compose
11+
12+
This method uses Docker Compose to start all necessary services with a single command. It is the fastest and easiest way to get the service running.
13+
14+
### 1.1. Access the Code
15+
16+
Clone the repository and navigate to the deployment directory:
17+
18+
```bash
19+
git clone https://github.com/opea-project/GenAIComps.git
20+
cd GenAIComps/comps/animation/deployment/docker_compose
21+
```
22+
23+
### 1.2. Deploy the Service
24+
25+
Choose the command corresponding to your target platform.
26+
27+
- **For Intel® Xeon® CPU:**
28+
29+
```bash
30+
docker compose -f compose.yaml up animation -d
31+
```
32+
33+
- **For Intel® Gaudi® 2 HPU:**
34+
```bash
35+
docker compose -f compose.yaml up animation-gaudi -d
36+
```
37+
38+
### 1.3. Validate the Service
39+
40+
Once the containers are running, you can validate the service. **Note:** Run these commands from the root of the `GenAIComps` repository.
41+
42+
```bash
43+
# Navigate back to the root directory if you are in the docker_compose folder
44+
cd ../../..
45+
46+
# Validate the Animation service endpoint
47+
export ip_address=$(hostname -I | awk '{print $1}')
48+
curl http://${ip_address}:9066/v1/animation -X POST \
49+
-H "Content-Type: application/json" \
50+
-d @comps/animation/src/assets/audio/sample_question.json
51+
```
52+
53+
The expected output will be a JSON object containing the path to the generated video file:
54+
55+
```json
56+
{ "wav2lip_result": ".../GenAIComps/comps/animation/src/assets/outputs/result.mp4" }
57+
```
58+
59+
The generated video `result.mp4` will be available in the `comps/animation/src/assets/outputs/` directory.
60+
61+
### 1.4. Clean Up the Deployment
62+
63+
To stop and remove the containers, run the following command from the `comps/animation/deployment/docker_compose` directory:
64+
65+
```bash
66+
docker compose down
67+
```
68+
69+
---
70+
71+
## 🚀 2. Manual Step-by-Step Deployment (Advanced)
72+
73+
This section provides detailed instructions for building the Docker images and running each microservice container individually.
74+
75+
### 2.1. Clone the Repository
76+
77+
If you haven't already, clone the repository and navigate to the root directory:
78+
79+
```bash
80+
git clone https://github.com/opea-project/GenAIComps.git
81+
cd GenAIComps
82+
```
83+
84+
### 2.2. Build the Docker Images
85+
86+
#### 2.2.1. Build Wav2Lip Server Image
87+
88+
- **For Intel® Xeon® CPU:**
89+
```bash
90+
docker build -t opea/wav2lip:latest -f comps/third_parties/wav2lip/src/Dockerfile .
91+
```
92+
- **For Intel® Gaudi® 2 HPU:**
93+
```bash
94+
docker build -t opea/wav2lip-gaudi:latest -f comps/third_parties/wav2lip/src/Dockerfile.intel_hpu .
95+
```
96+
97+
#### 2.2.2. Build Animation Server Image
98+
99+
```bash
100+
docker build -t opea/animation:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/animation/src/Dockerfile .
101+
```
102+
103+
### 2.3. Configure Environment Variables
104+
105+
Set the necessary environment variables for the containers.
106+
107+
- **For Intel® Xeon® CPU:**
108+
109+
```bash
110+
export ip_address=$(hostname -I | awk '{print $1}')
111+
export DEVICE="cpu"
112+
export WAV2LIP_PORT=7860
113+
export CHECKPOINT_PATH='/usr/local/lib/python3.11/site-packages/Wav2Lip/checkpoints/wav2lip_gan.pth'
114+
export PYTHON_PATH='/usr/bin/python3.11'
115+
```
116+
117+
- **For Intel® Gaudi® 2 HPU:**
118+
```bash
119+
export ip_address=$(hostname -I | awk '{print $1}')
120+
export DEVICE="hpu"
121+
export WAV2LIP_PORT=7860
122+
export CHECKPOINT_PATH='/usr/local/lib/python3.10/dist-packages/Wav2Lip/checkpoints/wav2lip_gan.pth'
123+
export PYTHON_PATH='/usr/bin/python3.10'
124+
```
125+
126+
### 2.4. Run the Microservice Containers
127+
128+
#### 2.4.1. Run Wav2Lip Microservice
129+
130+
- **For Intel® Xeon® CPU:**
131+
132+
```bash
133+
docker run --privileged -d --name "wav2lip-service" -p $WAV2LIP_PORT:$WAV2LIP_PORT --ipc=host \
134+
-w /home/user/comps/animation/src \
135+
-v $(pwd)/comps/animation/src/assets:/home/user/comps/animation/src/assets \
136+
-e PYTHON=$PYTHON_PATH \
137+
-e DEVICE=$DEVICE \
138+
-e CHECKPOINT_PATH=$CHECKPOINT_PATH \
139+
-e WAV2LIP_PORT=$WAV2LIP_PORT \
140+
opea/wav2lip:latest
141+
```
142+
143+
- **For Intel® Gaudi® 2 HPU:**
144+
```bash
145+
docker run --privileged -d --name "wav2lip-gaudi-service" -p $WAV2LIP_PORT:$WAV2LIP_PORT --runtime=habana --cap-add=sys_nice --ipc=host \
146+
-w /home/user/comps/animation/src \
147+
-v $(pwd)/comps/animation/src/assets:/home/user/comps/animation/src/assets \
148+
-e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none \
149+
-e PYTHON=$PYTHON_PATH \
150+
-e DEVICE=$DEVICE \
151+
-e CHECKPOINT_PATH=$CHECKPOINT_PATH \
152+
-e WAV2LIP_PORT=$WAV2LIP_PORT \
153+
opea/wav2lip-gaudi:latest
154+
```
155+
156+
#### 2.4.2. Run Animation Microservice
157+
158+
```bash
159+
docker run -d --name "animation-service" -p 9066:9066 --ipc=host \
160+
-e http_proxy=$http_proxy \
161+
-e https_proxy=$https_proxy \
162+
-e WAV2LIP_ENDPOINT=http://$ip_address:$WAV2LIP_PORT \
163+
opea/animation:latest
164+
```
165+
166+
### 2.5. Validate the Service
167+
168+
After starting both containers, test the animation service endpoint. Make sure you are in the root directory of the `GenAIComps` repository.
169+
170+
```bash
171+
# The ip_address variable should be set from step 2.3
172+
curl http://${ip_address}:9066/v1/animation -X POST \
173+
-H "Content-Type: application/json" \
174+
-d @comps/animation/src/assets/audio/sample_question.json
175+
```
176+
177+
You should see a successful response with the path to the output video.
178+
179+
### 2.6. Clean Up the Deployment
180+
181+
To stop and remove the containers you started manually, use the `docker stop` and `docker rm` commands.
182+
183+
- **For Intel® Xeon® CPU:**
184+
185+
```bash
186+
docker stop wav2lip-service animation-service
187+
docker rm wav2lip-service animation-service
188+
```
189+
190+
- **For Intel® Gaudi® 2 HPU:**
191+
```bash
192+
docker stop wav2lip-gaudi-service animation-service
193+
docker rm wav2lip-gaudi-service animation-service
194+
```

‎comps/animation/src/README.md‎

Lines changed: 18 additions & 133 deletions
Original file line numberDiff line numberDiff line change
@@ -2,146 +2,31 @@
22

33
The avatar animation model is a combination of two models: Wav2Lip and GAN-based face generator (GFPGAN). The Wav2Lip model is used to generate lip movements from an audio file, and the GFPGAN model is used to generate a high-quality face image from a low-quality face image. The avatar animation microservices takes an audio piece and a low-quality face image/video as input, fuses mel-spectrogram from the audio with frame(s) from the image/video, and generates a high-quality video of the face image with lip movements synchronized with the audio.
44

5-
# 🚀1. Start Microservice with Docker (option 1)
5+
## Table of contents
66

7-
## 1.1 Build the Docker images
7+
1. [Architecture](#architecture)
8+
2. [Deployment Options](#deployment-options)
9+
3. [Validated Configurations](#validated-configurations)
810

9-
### 1.1.1 Wav2Lip Server image
11+
## Architecture
1012

11-
```bash
12-
git clone https://github.com/opea-project/GenAIComps.git
13-
cd GenAIComps
14-
```
13+
The Avatar Animation service consists of two primary microservices:
1514

16-
- Xeon CPU
15+
- **Wav2Lip Server**: This microservice is the core engine for lip synchronization. It takes an audio file and a face image/video as input and generates a video where the lip movements match the provided audio. It can be deployed on both CPU and HPU.
16+
- **Animation Server**: This microservice acts as an orchestrator or gateway. It exposes a single endpoint for the user, receives the request, forwards it to the Wav2Lip server for processing, and then returns the final generated video to the user.
1717

18-
```bash
19-
docker build -t opea/wav2lip:latest -f comps/third_parties/wav2lip/src/Dockerfile .
20-
```
18+
## Deployment Options
2119

22-
- Gaudi2 HPU
20+
For detailed, step-by-step instructions on how to deploy the Avatar Animation microservice using Docker Compose on different Intel platforms, please refer to the deployment guide. The guide contains all necessary steps, including building images, configuring the environment, and running the service.
2321

24-
```bash
25-
docker build -t opea/wav2lip-gaudi:latest -f comps/third_parties/wav2lip/src/Dockerfile.intel_hpu .
26-
```
22+
| Platform | Deployment Method | Link |
23+
| ----------------- | ----------------- | ---------------------------------------------------------- |
24+
| Intel Xeon/Gaudi2 | Docker Compose | [Deployment Guide](../deployment/docker_compose/README.md) |
2725

28-
### 1.1.2 Animation server image
26+
## Validated Configurations
2927

30-
```bash
31-
docker build -t opea/animation:latest --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy -f comps/animation/src/Dockerfile .
32-
```
28+
The following configurations have been validated for the Avatar Animation microservice.
3329

34-
## 1.2. Set environment variables
35-
36-
- Xeon CPU
37-
38-
```bash
39-
export ip_address=$(hostname -I | awk '{print $1}')
40-
export DEVICE="cpu"
41-
export WAV2LIP_PORT=7860
42-
export ANIMATION_PORT=9066
43-
export INFERENCE_MODE='wav2lip+gfpgan'
44-
export CHECKPOINT_PATH='/usr/local/lib/python3.11/site-packages/Wav2Lip/checkpoints/wav2lip_gan.pth'
45-
export FACE="assets/img/avatar1.jpg"
46-
# export AUDIO='assets/audio/eg3_ref.wav' # audio file path is optional, will use base64str in the post request as input if is 'None'
47-
export AUDIO='None'
48-
export FACESIZE=96
49-
export OUTFILE="assets/outputs/result.mp4"
50-
export GFPGAN_MODEL_VERSION=1.4 # latest version, can roll back to v1.3 if needed
51-
export UPSCALE_FACTOR=1
52-
export FPS=10
53-
```
54-
55-
- Gaudi2 HPU
56-
57-
```bash
58-
export ip_address=$(hostname -I | awk '{print $1}')
59-
export DEVICE="hpu"
60-
export WAV2LIP_PORT=7860
61-
export ANIMATION_PORT=9066
62-
export INFERENCE_MODE='wav2lip+gfpgan'
63-
export CHECKPOINT_PATH='/usr/local/lib/python3.10/dist-packages/Wav2Lip/checkpoints/wav2lip_gan.pth'
64-
export FACE="assets/img/avatar1.jpg"
65-
# export AUDIO='assets/audio/eg3_ref.wav' # audio file path is optional, will use base64str in the post request as input if is 'None'
66-
export AUDIO='None'
67-
export FACESIZE=96
68-
export OUTFILE="assets/outputs/result.mp4"
69-
export GFPGAN_MODEL_VERSION=1.4 # latest version, can roll back to v1.3 if needed
70-
export UPSCALE_FACTOR=1
71-
export FPS=10
72-
```
73-
74-
# 🚀2. Run the Docker container
75-
76-
## 2.1 Run Wav2Lip Microservice
77-
78-
- Xeon CPU
79-
80-
```bash
81-
docker run --privileged -d --name "wav2lip-service" -p 7860:7860 --ipc=host -w /home/user/comps/animation/src -e PYTHON=/usr/bin/python3.11 -v $(pwd)/comps/animation/src/assets:/home/user/comps/animation/src/assets -e DEVICE=$DEVICE -e INFERENCE_MODE=$INFERENCE_MODE -e CHECKPOINT_PATH=$CHECKPOINT_PATH -e FACE=$FACE -e AUDIO=$AUDIO -e FACESIZE=$FACESIZE -e OUTFILE=$OUTFILE -e GFPGAN_MODEL_VERSION=$GFPGAN_MODEL_VERSION -e UPSCALE_FACTOR=$UPSCALE_FACTOR -e FPS=$FPS -e WAV2LIP_PORT=$WAV2LIP_PORT opea/wav2lip:latest
82-
```
83-
84-
- Gaudi2 HPU
85-
86-
```bash
87-
docker run --privileged -d --name "wav2lip-gaudi-service" -p 7860:7860 --runtime=habana --cap-add=sys_nice --ipc=host -w /home/user/comps/animation/src -v $(pwd)/comps/animation/src/assets:/home/user/comps/animation/src/assets -e HABANA_VISIBLE_DEVICES=all -e OMPI_MCA_btl_vader_single_copy_mechanism=none -e PYTHON=/usr/bin/python3.10 -e DEVICE=$DEVICE -e INFERENCE_MODE=$INFERENCE_MODE -e CHECKPOINT_PATH=$CHECKPOINT_PATH -e FACE=$FACE -e AUDIO=$AUDIO -e FACESIZE=$FACESIZE -e OUTFILE=$OUTFILE -e GFPGAN_MODEL_VERSION=$GFPGAN_MODEL_VERSION -e UPSCALE_FACTOR=$UPSCALE_FACTOR -e FPS=$FPS -e WAV2LIP_PORT=$WAV2LIP_PORT opea/wav2lip-gaudi:latest
88-
```
89-
90-
## 2.2 Run Animation Microservice
91-
92-
```bash
93-
docker run -d -p 9066:9066 --ipc=host --name "animation-service" -e http_proxy=$http_proxy -e https_proxy=$https_proxy -e WAV2LIP_ENDPOINT=http://$ip_address:7860 opea/animation:latest
94-
```
95-
96-
# 🚀3. Start Microservice with Docker Compose
97-
98-
Alternatively, you can also start the Animation microservice with Docker Compose.
99-
100-
- Xeon CPU
101-
102-
```bash
103-
cd comps/animation/deployment/docker_compose
104-
docker compose -f compose.yaml up animation -d
105-
106-
```
107-
108-
- Gaudi2 HPU
109-
110-
```bash
111-
cd comps/animation/deployment/docker_compose
112-
docker compose -f compose.yaml up animation-gaudi -d
113-
```
114-
115-
# 🚀4. Validate Microservice
116-
117-
Once microservice starts, user can use below script to validate the running microservice.
118-
119-
## 4.1 Validate Wav2Lip service
120-
121-
```bash
122-
cd GenAIComps
123-
python3 comps/third_parties/wav2lip/src/check_wav2lip_server.py
124-
```
125-
126-
## 4.2 Validate Animation service
127-
128-
```bash
129-
cd GenAIComps
130-
export ip_address=$(hostname -I | awk '{print $1}')
131-
curl http://${ip_address}:9066/v1/animation -X POST -H "Content-Type: application/json" -d @comps/animation/src/assets/audio/sample_question.json
132-
```
133-
134-
or
135-
136-
```bash
137-
cd GenAIComps
138-
python3 comps/third_parties/wav2lip/src/check_animation_server.py
139-
```
140-
141-
The expected output will be a message similar to the following:
142-
143-
```bash
144-
{'wav2lip_result': '....../GenAIComps/comps/animation/src/assets/outputs/result.mp4'}
145-
```
146-
147-
Please find "comps/animation/src/assets/outputs/result.mp4" as a reference generated video.
30+
| **Deploy Method** | **Core Models** | **Platform** |
31+
| ----------------- | --------------- | ----------------- |
32+
| Docker Compose | Wav2Lip, GFPGAN | Intel Xeon/Gaudi2 |

0 commit comments

Comments
 (0)