Skip to content

README.md

Latest commit

 

History

History
291 lines (205 loc) · 8.33 KB

README.md

File metadata and controls

291 lines (205 loc) · 8.33 KB
Raw

Dreamverse

Dreamverse is the FastVideo realtime video generation & editing platform. It lives in this monorepo under apps/dreamverse/.

Deploy on: local GPU · self-hosted B200 (SSH) · Docker · Modal

Install Dreamverse

You can install Dreamverse using one of the methods below.

Method 1: With uv pip

pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
uv pip install "fastvideo[dreamverse]"

Method 2: From source

git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

pip install --upgrade pip
pip install uv
uv venv .venv --python 3.12
source .venv/bin/activate
uv pip install -e ".[dreamverse]"

Method 3: Using Docker

git clone https://github.com/hao-ai-lab/FastVideo.git
cd FastVideo

apps/dreamverse/docker/docker_build.sh

See apps/dreamverse/docker/README.md for Docker build and run option details.

Optional: Building FFmpeg For Better Performance

For full streaming performance in a non-Docker install, build a custom FFmpeg binary from a FastVideo source checkout. The command below is repo-relative, so run it from the repository root:

bash apps/dreamverse/scripts/install_native_ffmpeg.sh

The installer supports Linux x86_64 and aarch64. It prefers conda-forge triplet compilers when those commands are on PATH, otherwise it falls back to system gcc/g++ (plain venv). On x86_64, x264's hand-tuned SIMD also requires nasm; install via whichever path fits your host:

sudo apt install nasm                       # Debian/Ubuntu
conda install -c conda-forge nasm           # inside an active conda env

No sudo and no conda? Build nasm from source (~30s, installs into $HOME):

(
  mkdir -p "$HOME/src" "$HOME/opt" && cd "$HOME/src"
  curl -fsSL -O https://www.nasm.us/pub/nasm/releasebuilds/2.16.03/nasm-2.16.03.tar.gz
  tar -xf nasm-2.16.03.tar.gz && cd nasm-2.16.03
  ./configure --prefix="$HOME/opt/nasm" && make -j"$(nproc)" && make install
)
export PATH="$HOME/opt/nasm/bin:$PATH"      # add to ~/.bashrc to persist

The installer writes to ~/opt/ffmpeg-native/ and emits apps/dreamverse/scripts/ffmpeg-env.sh. Source it before starting the backend so Dreamverse uses the custom FFmpeg binary:

source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server

Docker images already run this FFmpeg build during image creation and source the generated environment file at container startup.

Launch Dreamverse

Start the backend with the installed Dreamverse commands:

dreamverse-server --port 8009
dreamverse-mock-server --port 8009

Expect a slow first boot. With torch.compile and startup warmup enabled (the default), the backend compiles the segment 1 and segment 2 inference paths before it reports ready — this can take tens of minutes on a cold cache, regardless of how you deploy (local, server, Docker, or Modal). /healthz responds as soon as the process is up; /readyz stays 503 until warmup finishes. For a faster, uncompiled startup while testing, set FASTVIDEO_ENABLE_STARTUP_WARMUP=0 before starting the backend.

Frontend Setup

Install the web dependencies once from the FastVideo checkout:

cd apps/dreamverse/web
npm ci

The frontend package uses package-lock.json; use npm for installs and scripts.

Quick Start: Local GPU

Start Backend

Export the API keys used for prompt rewrite and prompt enhancement:

export CEREBRAS_API_KEY=...
export GROQ_API_KEY=...

If you built the optional native FFmpeg binary above, source its environment file in the same shell before starting the backend:

source apps/dreamverse/scripts/ffmpeg-env.sh
dreamverse-server --host 0.0.0.0 --port 8009

The Dreamverse backend defaults to 0.0.0.0:8009 and starts one GPU worker on the first visible GPU by default.

Check Readiness

In another shell, verify that the backend process is alive:

curl http://localhost:8009/healthz

Then wait for GPU workers and startup warmup to finish:

curl http://localhost:8009/readyz

You can also run the same readiness path with:

BACKEND_HOST=localhost BACKEND_PORT=8009 apps/dreamverse/scripts/smoke_local.sh

If a backend is already running and you only want the script to probe it:

DREAMVERSE_SMOKE_START_BACKEND=0 apps/dreamverse/scripts/smoke_local.sh

Start Frontend

Start the frontend:

cd apps/dreamverse/web
BACKEND_HOST=localhost BACKEND_PORT=8009 npm run dev

Open http://localhost:5299.

Server B200 deployment (SSH)

Deploying on a remote GPU host (for example a B200 box) is a local install run over SSH, plus a few server-specific concerns. Two paths:

Option A: Native (source install)

SSH in, then follow Install → From source and (recommended) Building FFmpeg, then start the backend as in Quick Start: Local GPU. For a remote host, a few things differ from localhost:

  • Bind all interfaces: dreamverse-server --host 0.0.0.0 --port 8009.
  • Point the frontend/client at the host: BACKEND_HOST=<b200-host> BACKEND_PORT=8009 npm run dev.
  • Keep the backend alive across SSH sessions (tmux / systemd / nohup).
  • Expose / firewall port 8009, or front it with a reverse proxy + auth.

Option B: Docker (on the server)

SSH in, then follow Install → Using Docker and the run steps in docker/README.md:

CEREBRAS_API_KEY="<key>" GROQ_API_KEY="<key>" apps/dreamverse/docker/docker_run.sh

Quick Start: Mock Backend (For UI development)

The mock server emulates the Dreamverse backend protocol and streams a synthetic FFmpeg-generated fMP4 clip, so the frontend can run without a GPU.

dreamverse-mock-server --latency 200 --port 8009

Tests

Run the focused backend tests that validate local startup wiring, config, GPU selection, and mock-server behavior:

pytest apps/dreamverse/dreamverse/tests/test_config.py \
  apps/dreamverse/dreamverse/tests/test_entrypoints.py \
  apps/dreamverse/dreamverse/tests/test_gpu_pool.py \
  apps/dreamverse/dreamverse/tests/test_mock_server.py -q

Run the broader Dreamverse backend suite:

pytest apps/dreamverse/dreamverse/tests -q

Run the frontend tests:

cd apps/dreamverse/web
npm test

Run the frontend e2e tests:

cd apps/dreamverse/web
npm run e2e

Troubleshooting

dreamverse-server exits with an install hint

  • install the Dreamverse extra with uv pip install -e ".[dreamverse]" from a source checkout, or uv pip install "fastvideo[dreamverse]" from PyPI.

Prompt-provider environment variable errors

  • set CEREBRAS_API_KEY
  • set GROQ_API_KEY
  • direct dreamverse-server launches do not source ~/.env; export the keys in the shell or use the bundled launch scripts, which source ~/.env.

/readyz stays at 503

  • wait for model loading and startup warmup to finish
  • confirm a compatible CUDA GPU is visible to the process
  • check backend logs for worker startup or warmup failures
  • for a startup/debug pass without warmup, set FASTVIDEO_ENABLE_STARTUP_WARMUP=0 before starting the backend

Only one GPU is used

  • this is the default local behavior
  • set FASTVIDEO_GPU_COUNT=<N> to start N GPU worker subprocesses inside one backend instance
  • set FASTVIDEO_GPU_COUNT=all to start one worker for every visible GPU
  • use CUDA_VISIBLE_DEVICES first if you need to pin the visible GPU set

Frontend cannot connect to backend

  • confirm the backend is running on 8009; if not, point the frontend at it with BACKEND_HOST=<host> BACKEND_PORT=<port> npm run dev
  • confirm http://localhost:8009/healthz responds before starting the frontend
  • confirm http://localhost:8009/readyz returns 200 before clicking Generate
  • use apps/dreamverse/scripts/smoke_local.sh for a repeatable local startup check

Mock backend fails during startup

  • install FFmpeg or set FASTVIDEO_FFMPEG_BIN to an FFmpeg binary
  • for non-mock local GPU streaming performance, use the native FFmpeg installer described above

Notes

Dreamverse owns its backend app under apps/dreamverse/dreamverse/. It expects dreamverse-server, not fastvideo serve.