Skip to content

TTS_dojo_guide.md

Latest commit

 

History

History
64 lines (47 loc) · 7.27 KB

TTS_dojo_guide.md

File metadata and controls

64 lines (47 loc) · 7.27 KB
Raw

TTS Dojo

  • The TTS Dojo is a set of shell scripts that use Piper TTS to produce custom text-to-speech models, either by finetuning pretrained Piper TTS checkpoint files or training models from scratch.
  • Piper's training services are provided via a docker container called textymcspeechy-piper, which can either be downloaded from dockerhub or built locally.
  • Voice dojos automatically start the textymcspeechy-piper docker image by calling TextyMcSpeechy/run_container.sh and automatically stop it when training ends.
  • If you have set up custom pronunciation rules in tts_dojo/ESPEAK_RULES, these can be configured to be activated automatically when your container runs. A guide can be found here

About the training process

  • The quick start guide contains everything you need to know about getting your first training session running
  • Be aware the provided training scripts delete all data stored in <VOICE_NAME>_dojo/training_folder/lightning_logs at the beginning of each training run.
  • When training starts, the training scripts run in a multi-window environment provided by tmux. The tmux session is named training.
  • The training session can be shut down from any terminal window with the command tmux kill-session if there is a problem and the normal ways of shutting the session down are unavailable.
  • If your training session crashes you may need to manually bring down the textymcspeechy-piper docker image. Check its status by running docker ps
  • You can shut the docker container down manually by either:
    • navigating to the main TextyMcSpeechy folder and running ./stop_container.sh
    • running docker stop textymcspeechy-piper from any terminal window.
  • The layout of the training panes isn't always ideal the first time you run a training session. You can drag the boundaries between panes around with your mouse to adjust them to your liking. The layout can be saved by clicking on the "CONTROL CONSOLE" pane and pressing t. This layout will be applied automatically on future training runs.

What do the different windows in the training dojo do?

image

  1. PIPER TRAINING RAW OUTPUT This pane contains all of the text generated by piper as it trains your model. I generally recommend making this pane 2 rows tall since most of the information provided here isn't very useful. There are often many warnings that appear in this pane. Most of them can be ignored.

    • The best indicator that your training session is working properly is when rank_zero_warn ( appears at the bottom of this pane. It is normal for this to appear for several minutes as training is initializing.
    • You may see a warning about a low number of workers being a bottleneck. Currently the only way to resolve this is to modify piper's source code.
    • If you see an error related to zip files here, this usually means your starting checkpoint file is corrupted. Either restart training or delete the highest epoch checkpoint in the voice_checkpoints directory.

  2. TENSORBOARD SERVER This pane runs a web server that lets you view graphs related to training progress. Open http://localhost:6006 in your web browser if you want to see them. When the graph for loss_disc_all levels off, your model is probably almost ready. It isn't necessary to use this at all during training but I provide it for those who prefer a more quantitative approach than "listening to a model and seeing if it sounds good" (my preferred method).

  3. TTS MODEL EXPORTER This pane contains piper's output when it converts a checkpoint file into a .onnx file. It can be reduced to a single row high since it produces almost no output during normal operation.

  4. CHECKPOINT GRABBER image

    • The CHECKPOINT GRABBER pane runs a script which allows you to periodically save one of the checkpoint files that Piper generates during training and automatically convert it into a text-to-speech model.
    • This allows you to decide when your model is done training by listening to which checkpoint's version of the voice sounds the best.
    • Beware that leaving this tool unattended for a long time or saving checkpoint files frequently could quite easily fill your entire hard drive -- each checkpoint file is over 800MB.
    • You can set up an automatic shutdown if your available storage falls below a certain threshold by editing the global SETTINGS.txt file in tts_dojo/DOJO_CONTENTS/scripts/SETTINGS.txt before creating a new voice dojo.
    • You can manually save the most recent checkpoint file and convert it to a Piper voice model by pressing s while this pane is selected. This can be useful if you need to shut down a training session and resume it later. run_training.sh will automatically prompt you to resume training from the highest epoch checkpoint you have saved.
    • Since checkpoint files are so large, by default the checkpoint grabber only saves 1 out of every 25 checkpoints that piper generates. You can change how often checkpoints are automatically saved by pressing i to save them less often and and d to save them more often. You can also turn automatic saving off and on by pressing t.

  5. CONTROL CONSOLE image

    • This pane displays information about the amount of storage your training session is using, and also provides several controls when it is selected. Select the CONTROL CONSOLE pane by clicking it (or navigating to it with <CTRL>-B followed by arrow keys) if you don't have a mouse, then:
    • To shut down training: press q.
    • To save the current tmux window layout: press t
    • To restore a previous tmux window layout: press r

  6. VOICE TESTER
    image

    • This pane allows you to hear what the voice associated with a saved checkpoint file sounds like.
    • After at least one checkpoint has been saved and exported, a list of voice files will appear in this window.
    • Select this window and use the arrow keys to highlight the version of the voice you want to hear, then press s to have it speak the text in the "Text to say:" field.
    • The number following the underscore is the epoch number of the corresponding checkpoint file saved in <VOICE_NAME_dojo>/voice_checkpoints
    • Any voice that appears in the voice tester pane is ready to be used in your Piper projects.
    • Your finished Piper voices are stored in subfolders of <VOICE_NAME_dojo>/tts_voices
    • A Piper voice consists of both an .onnx file and a .onnx.json file
    • Note: the latest version of the TTS dojo scripts name the voice files according to Piper's expected naming conventions. Voices trained with earlier versions of TextyMcSpeechy or voices that have been manually renamed may need to be modified using this guide in order to make them work properly with apps such as Home Assistant.