Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,14 @@ More detailed instructions for setting up the [client](src/client/react/README.m
docker compose up --build
```

This default Docker setup builds the server for CPU / non-GPU deployment.

To build with GPU-oriented extras and request a GPU-enabled container runtime:

```bash
docker compose -f docker-compose.yaml -f docker-compose.gpu.yaml up --build
```

#### 3b. Run manually in two different tabs of your terminal (recommended)

- **Start the server:**
Expand All @@ -117,6 +125,16 @@ docker compose up --build
python main.py
```

For a GPU-oriented install on Linux, use:
```bash
pip install -r requirements.gpu.txt
```

To force local inference onto CPU even on a machine with an accelerator, set:
```bash
export RIVERST_COMPUTE_DEVICE=cpu
```

- **Start the client:**
```bash
cd src/client/react
Expand Down
8 changes: 8 additions & 0 deletions docker-compose.gpu.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
services:
server:
build:
args:
RIVERST_DEPLOYMENT_TARGET: gpu
environment:
RIVERST_COMPUTE_DEVICE: auto
gpus: all
4 changes: 4 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,15 @@ services:
build:
context: ./src/server
dockerfile: Dockerfile
args:
RIVERST_DEPLOYMENT_TARGET: ${RIVERST_DEPLOYMENT_TARGET:-cpu}
container_name: riverst_server
ports:
- "7860:7860"
env_file:
- ./src/server/.env
environment:
RIVERST_COMPUTE_DEVICE: ${RIVERST_COMPUTE_DEVICE:-auto}
volumes:
- ./src/server/sessions:/app/sessions

Expand Down
43 changes: 43 additions & 0 deletions docs/gpu-cpu-deployment-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# GPU and Non-GPU Deployment Plan

## Goal

Allow Riverst to deploy in either of these modes without code changes:

- [x] CPU / non-GPU deployment
- [x] GPU-capable deployment

## Findings

After reviewing the repository, no server function strictly requires a GPU to execute. The GPU-sensitive paths are all local-model or local-inference paths where acceleration improves latency or throughput.

### GPU-sensitive functions and classes

- [x] `src/server/bot/utils/device_utils.py:get_best_device`
- [x] `src/server/bot/processors/audio/resampling_helper.py:AudioResamplingHelper._torchaudio_resample`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:predict_phonemes_from_waveform`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:load_cupe_model`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:LipsyncProcessor.__init__`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:LipsyncProcessor._warm_up`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:LipsyncProcessor._preprocess_audio`
- [x] `src/server/bot/processors/speech/lipsync_processor.py:LipsyncProcessor._run_lipsync`
- [x] `src/server/bot/core/component_factory.py:BotComponentFactory._build_stt_service`
- [x] `src/server/bot/core/component_factory.py:BotComponentFactory._build_tts_service`
- [x] `src/server/bot/processors/video/processor.py:VideoProcessor.__init__`
- [x] `src/server/bot/processors/video/processor.py:VideoProcessor._run_pose_in_background`
- [x] `src/server/bot/processors/audio/analyzer.py:AudioAnalyzer.analyze_audio`

## Decisions

- [x] Add a runtime device policy via `RIVERST_COMPUTE_DEVICE=auto|cpu`
- [x] Add a Docker build target via `RIVERST_DEPLOYMENT_TARGET=cpu|gpu`
- [x] Add a GPU compose override for `gpus: all`
- [x] Keep CPU deployments functional when ONNX export dependencies are unavailable
- [x] Add a small test suite for the runtime device policy

## Implementation Notes

- CPU deployments now default to `requirements.txt`
- GPU deployments install `requirements.gpu.txt`
- Runtime device selection is centralized in `bot.utils.device_utils`
- Pose processing falls back to the PyTorch YOLO model if ONNX export is unavailable
Loading