[Bug] IndexTTS2 /v1/audio/speech returns 500 at WAV save: libtorchcodec cannot load in v2.12.0 Docker

[kvenLin]

System Info / 系统信息

• Host: Windows + Docker Desktop
• Container OS: Ubuntu 22.04.5 LTS
• GPU: NVIDIA GeForce RTX 5070 Ti, 16 GB
• NVIDIA driver: 591.86
• Image: xprobe/xinference:v2.12.0
• Python: 3.12
• PyTorch: 2.11.0+cu130
• TorchAudio: 2.11.0+cu130
• TorchCodec: 0.10.0
• FFmpeg CLI: 6.1.2
• SoundFile: 0.14.0

The IndexTTS2 model reaches READY and inference itself completes. The failure happens only when saving the generated waveform.

Running Xinference with Docker? / 是否使用 Docker 运行 Xinference？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

v2.12.0

The command used to start Xinference / 用以启动 Xinference 的命令

services:
  xinference:
    image: xprobe/xinference:v2.12.0
    gpus: all
    shm_size: "8gb"
    ports:
      - "127.0.0.1:9997:9997"
    environment:
      XINFERENCE_HOME: /data
      XINFERENCE_ENABLE_VIRTUAL_ENV: "1"
    command: ["xinference-local", "-H", "0.0.0.0"]

Reproduction / 复现过程

1. Start the v2.12.0 Docker image.
2. Launch IndexTTS2 and wait until the replica is READY.
3. Open its launch UI, provide a valid WAV speaker prompt and text such as 你好测试一下.
4. Generate speech, or call POST /v1/audio/speech.
5. IndexTTS2 runs inference, reaches the final "saving audio..." stage, then the endpoint returns HTTP 500.

Relevant call path:

xinference/model/audio/indextts2.py::speech
  -> thirdparty/indextts/infer_v2.py::infer
  -> torchaudio.save(output_path, wav.type(torch.int16), sampling_rate)
  -> torchaudio.save_with_torchcodec
  -> import torchcodec.encoders.AudioEncoder
  -> Could not load libtorchcodec

The main error is:

RuntimeError: Could not load libtorchcodec

FFmpeg version 6:
OSError: libavutil.so.58: cannot open shared object file: No such file or directory

FFmpeg version 4:
OSError: libtorchcodec_core4.so: undefined symbol:
_ZN3c1013MessageLoggerC1EPKciib

ldd /usr/local/lib/python3.12/dist-packages/torchcodec/libtorchcodec_core6.so reports:

libtorch.so => not found
libc10.so => not found
libavutil.so.58 => not found
libavcodec.so.60 => not found
libavformat.so.60 => not found
libtorch_cpu.so => not found

The container's ffmpeg -version command works, but TorchCodec cannot resolve the corresponding shared libraries. There also appears to be a TorchCodec/PyTorch ABI mismatch for the fallback core library.

Full request ends with:

POST /v1/audio/speech HTTP/1.1" 500

Expected behavior / 期待表现

IndexTTS2 should return the generated WAV/audio response successfully after inference.

Analysis / 建议

The generated waveform already exists before the failure. This issue is specific to the output serialization path. A narrowly scoped fix would be to avoid TorchCodec for this IndexTTS2 WAV write and use soundfile.write (already installed in the image), or ensure the image ships a TorchCodec build and FFmpeg shared-library paths compatible with torch==2.11.0+cu130.

For example, replacing the final torchaudio.save call with an int16 SoundFile write avoids changing the inference path and avoids impacting other models that may depend on TorchCodec.

Related issues

• 升级到v2.9.0后，加载Eembedding和reranker模型报错 Could not load libtorchcodec. #4960 reports a generic Could not load libtorchcodec problem for embedding/reranker models on v2.9.0, but does not cover IndexTTS2 inference succeeding and failing specifically at WAV serialization on v2.12.0.
• 运行 IndexTTS2 缺少依赖 #4586 covers the separate IndexTTS2 No module named 'tn.english' dependency problem. That dependency was fixed locally before reproducing this output-save failure.

[qinxuye]

let's see if the upcoming v3.0 could solve this issue.

[kvenLin]

Tested the released xprobe/xinference:v3.0.0 image as requested.

Environment:

• Image: xprobe/xinference:v3.0.0
• Image digest: sha256:24fccb80568f65772a34eeab80e65320a0cd713b4b42c4c746d10b1c43f1375f
• Xinference: 3.0.0+0.g8ca6a625.dirty
• GPU: RTX 5070 Ti 16 GB
• Driver: 591.86
• PyTorch: 2.11.0+cu130
• TorchAudio: 2.11.0+cu130
• TorchCodec: 0.14.0+cu130

Unfortunately IndexTTS2 does not launch out of the box on v3.0.0, so I could not reach the original WAV serialization step.

The launch failures occur sequentially:

ModuleNotFoundError: No module named 'librosa'

After manually installing librosa==0.11.0 and soundfile==0.13.1 into the per-model venv:

ModuleNotFoundError: No module named 'einops'

After manually installing the next direct imports (einops, omegaconf, sentencepiece, and textstat):

ModuleNotFoundError: No module named 'ffmpy'

This appears to be an incomplete IndexTTS2 virtualenv dependency declaration rather than only a stale environment issue:

• xinference/model/audio/model_spec.json declares only:
  transformers==4.52.1, system torch/numpy, json5, munch, matplotlib, flatten_dict, julius, tensorboard, randomname, and argbind.
• The bundled thirdparty/indextts/infer_v2.py directly imports librosa, einops, omegaconf, sentencepiece, etc.
• The new slim v3 GPU image does not provide librosa or soundfile globally.
• The upstream IndexTTS2 pyproject.toml declares a much larger dependency set.

The first test reused the persisted per-model venv from v2.12.0, but v3's own model spec still does not declare the missing packages and the v3 global environment does not contain them, so a fresh v3 model venv appears likely to hit the same failures.

Also, the original save path is still present in v3.0.0:

xinference/thirdparty/indextts/infer_v2.py:878
torchaudio.save(output_path, wav.type(torch.int16), sampling_rate)

Therefore v3.0.0 does not currently provide a clean confirmation that the original TorchCodec failure is fixed: IndexTTS2 is blocked earlier by missing runtime dependencies, and the original torchaudio.save call remains unchanged.

[github-actions]

This issue is stale because it has been open for 14 days with no activity.