[Bug] tts_with_vc / tts_with_vc_to_file fails with XTTS: multi-speaker validation ignores speaker_wav

[rj25031]

Describe the bug

When using XTTS (tts_models/multilingual/multi-dataset/xtts_v2), the helper methods
tts_with_vc() and tts_with_vc_to_file() fail with:

ValueError: Model is multi-speaker but no speaker is provided.

This happens even when speaker_wav is provided, which should be sufficient for voice cloning with XTTS.

After inspecting the source, this appears to be a logic bug in how tts_with_vc internally calls tts_to_file.

To Reproduce

from TTS.api import TTS
import torch

tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda")

tts.tts_with_vc_to_file(
text="This is a test to reproduce.",
language="en",
speaker_wav="sample.wav",
file_path="out.wav"
)

Expected behavior

ValueError: Model is multi-speaker but no speaker is provided.

Logs

Traceback (most recent call last):
  File "/home/rupesh/voice_clone/TTS/md.py", line 30, in <module>
    tts.tts_with_vc_to_file(
  File "/home/rupesh/voice_clone/TTS/TTS/api.py", line 455, in tts_with_vc_to_file
    wav = self.tts_with_vc(
  File "/home/rupesh/voice_clone/TTS/TTS/api.py", line 415, in tts_with_vc
    self.tts_to_file(
  File "/home/rupesh/voice_clone/TTS/TTS/api.py", line 332, in tts_to_file
    self._check_arguments(speaker=speaker, language=language, speaker_wav=speaker_wav, **kwargs)
  File "/home/rupesh/voice_clone/TTS/TTS/api.py", line 227, in _check_arguments
    raise ValueError("Model is multi-speaker but no `speaker` is provided.")
ValueError: Model is multi-speaker but no `speaker` is provided.

Environment

OS: Linux (Ubuntu)
Python: 3.10
TTS version: GitHub main (XTTS v2)
GPU: CUDA (also reproducible on CPU)(Nvidia RTX 3090)
torch installed using pip 
cuda version 11.8

Additional context

No response

[eginhard]

XTTS can already do voice cloning, so just use tts_to_file() as shown in the documentation: https://coqui-tts.readthedocs.io/en/latest/models/xtts.html#tts-api

from TTS.api import TTS

tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2").to("cuda")

tts.tts_to_file(
  text="This is a test to reproduce.",
  language="en",
  speaker_wav="sample.wav",
  file_path="out.wav"
)

I'd recommend to use our fork (available via pip install coqui-tts), which will show a better error message in this case. This repo hasn't been maintained for 2 years.

[rj25031]

@eginhard Thanks for the clarification.

Just to clarify the intent of the report: this is not about XTTS voice cloning itself (tts_to_file with speaker_wav works as expected), but specifically about the helper methods tts_with_vc() / tts_with_vc_to_file() that still exist in this repository.

Those helpers currently fail deterministically with XTTS because speaker_wav is not forwarded to tts_to_file(), which causes _check_arguments() to raise for multi-speaker models.

I understand this repository is no longer actively maintained, so it’s completely fine to close this as “won’t fix” here. My main goal was to document the issue and root cause, since the API surface still exposes these helpers and users can easily run into this error.

For completeness, I’ve also opened a pull request with a minimal fix (forwarding speaker_wav) in case it’s useful as a reference, even if it’s not merged.

If this functionality is expected to live in the maintained fork instead, would it be appropriate to open a corresponding issue or PR there?

Thanks for the pointer to the maintained repository.

[eginhard]

For completeness, I’ve also opened a pull request with a minimal fix (forwarding speaker_wav) in case it’s useful as a reference, even if it’s not merged.

If this functionality is expected to live in the maintained fork instead, would it be appropriate to open a corresponding issue or PR there?

The fork already shows a more helpful warning message in that case, so no change is needed there:

UserWarning: This TTS model directly supports voice cloning, for better quality
call it with tts/tts_to_file(..., speaker_wav=...) instead.