Tortoise with RTX5090

[Vode1969]

I ran Tortoise with RTX4090, upgraded to RTX5090, but I can't get my head around how to get Tortoise to work. I installed PyTorch nightly, no luck. Using CPU works, but really slow as expected. Any hints appriciated.

[Vode1969]

Ok, it seems even passing tons of stuff to CPU before GPU doesn't work. CUDA sm_120 is a must. Maybe in the future someone compiles PyTorch for it.

[quasiblob]

There is already *now Pytorch 2.7.0, which has option for CUDA 12.8 (which is AFAIK required for 50 series RTX GPUs).

I've installed these torch packages, but so far I haven't had luck getting all the requirements installed, install fails and then there are dependencies with conflicting requirements.

[quasiblob]

OK, seems like I did get it installed with Pytorch 2.7 (CUDA 12.8) version.

Seems like setup.py gives several errors about packages not getting installed, also, conflicts when installing requirements. I fiddled with package versions and and got the do_tts.py to generate audio clips. Then I created new requirements.txt. After this, I cloned a fresh copy of the repo and created a venv, instead of conda environment.

I have no idea if it is ok to change some package versions or remove their version number to make pip happy, but at least it worked so far.

Now I can at least run the do_tts.py test script, and it successfully generates three sound clips! Not sure if it is using cuda like it should, but at least the venv has correct torch cuda versions installed, do_tts.py generation takes like ~30 seconds.

Too bad many tortoise frontends have similar issues, I can't get those installed.

[Vode1969]

I can confirm Tortoise working with the RTX5090. Required some additional tweaking (mostly updating other packages), but the main thing is that it's working. Some features can't be used, like cvvp-amount, but the basics is functional.

[joeysaladino]

Hello, I created the following dockerfile that drops the conda dependency, and adds support for Cuda 12.8. This is working well with my Nvidia RTX 5070 Ti.

Dockerfile:

FROM nvidia/cuda:12.8.0-base-ubuntu22.04

ENV DEBIAN_FRONTEND=noninteractive \
    HOME=/root \
    PIP_CACHE_DIR=/root/.cache/pip \
    TORTOISE_MODELS_DIR=/root/tortoise-tts/build/lib/tortoise/models \
    VENV_PATH=/opt/venv

# install prereqs
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      software-properties-common && \
    rm -rf /var/lib/apt/lists/* && \
    add-apt-repository ppa:deadsnakes/ppa

# install system & Python3.9 + venv support
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      git \
      wget \
      build-essential \
      ca-certificates \
      python3.9 \
      python3.9-venv \
      python3-pip && \
    rm -rf /var/lib/apt/lists/*

# create & activate virtualenv
RUN python3.9 -m venv $VENV_PATH && \
    $VENV_PATH/bin/pip install --upgrade pip setuptools wheel

ENV PATH="$VENV_PATH/bin:$PATH"

WORKDIR /app
COPY . .

# install pure-Python deps, then CUDA-12.8 PyTorch wheels, then your package
RUN pip install numba inflect progressbar einops rotary_embedding_torch librosa unidecode && \
    pip install --extra-index-url https://download.pytorch.org/whl/cu128 \
      torch==2.7.0+cu128 \
      torchvision==0.22.0+cu128 \
      torchaudio==2.7.0+cu128 && \
    pip install transformers==4.31.0 && \
    python setup.py install

You can build it with:

docker build . -t tts:cuda-12.8

Then run the container like so:

docker run --gpus all \
    -e TORTOISE_MODELS_DIR=/models \
    -v /path/to/tortoise-tts/models:/models \
    -v /path/to/tortoise-tts/results:/results \
    -v /mnt/user/data/.cache/huggingface:/root/.cache/huggingface \
    -v /root:/work \
    -it tts:cuda-12.8

Hope this is helpful for others!