Voice Trainer

Moroccan Darija TTS training pipeline based on FastPitch + HiFi-GAN, using the DODa dataset.

What This Repo Does

• Downloads and formats DODa audio/text data
• Merges datasets and ensures WAV-ready training data
• Extracts F0 (pitch) features
• Generates a training config automatically
• Fine-tunes FastPitch + HiFi-GAN
• Runs checkpoint-based inference tests

Requirements

• Linux environment recommended (local Linux, cloud VM, Vast, etc.)
• Python 3.10+ (3.12 supported in current workflow)
• NVIDIA GPU + CUDA
• ffmpeg
• Hugging Face account with access to DODa

Install ffmpeg:

sudo apt update && sudo apt install -y ffmpeg

Quickstart

1) Clone

git clone --recurse-submodules https://github.com/K-BEL/Voice_trainer.git
cd Voice_trainer

--recurse-submodules is required to pull models/tts/tts-arabic-pytorch.

2) Staged Setup (Recommended)

Each stage is independent so you can review the dataset before committing to a long training run:

# Stage 1 — Install Python packages + pretrained weights (once per instance)
bash scripts/setup-deps.sh

# Stage 2 — Download + format the DODa dataset
bash scripts/download-dataset.sh

Manually review the raw dataset before continuing:

datasets/doda-dataset/doda/data.csv     ← spot-check audio/caption pairs
datasets/doda-dataset/doda/audios/      ← listen to a few files

Delete any rows where the caption does not match the audio.

# Run the interactive review tool (plays 20 random samples)
python3 scripts/review_dataset.py \
    --dataset-dir datasets/doda-dataset/doda \
    --n 20
# Controls: Enter/k = keep | r = remove | q = quit
# Rejected rows are removed from data.csv automatically.

Then continue:

# Stage 3 — Merge, convert to WAV, extract F0, generate config
bash scripts/prepare-dataset.sh

# Stage 4 — Train
bash models/tts/src/finetune.sh

One-shot Setup (Automated)

Skips the manual review step — runs all stages in sequence:

bash scripts/setup_tts_doda.sh

Common options:

# Skip dataset download (dataset already exists)
bash scripts/setup_tts_doda.sh --skip-download

# Skip pretrained model download
bash scripts/setup_tts_doda.sh --skip-pretrained

# Customize training length at config-generation time
bash scripts/setup_tts_doda.sh --epochs 200 --save-interval 1000

Activate Environment Manually

The setup script creates a virtual environment at ./venv. To activate it for ad-hoc commands:

source venv/bin/activate

Train

bash models/tts/src/finetune.sh

Outputs:

• Checkpoints: models/tts/checkpoints/
• TensorBoard logs: models/tts/logs/

Test Checkpoints

Generate test audio for all .pth checkpoints in a directory:

bash models/tts/src/test-ckpts.sh models/tts/checkpoints

Generated audio is saved under:

• models/tts/results/<checkpoint_name>/

Typical Workflow

1. Run setup once
2. Start training
3. Let it run for a while
4. Test checkpoints periodically (or use the Gradio demo)
5. Best checkpoints are saved automatically as best_model.pth

Interactive Demo

Test any checkpoint interactively in your browser:

bash models/tts/src/demo.sh models/tts/checkpoints

Then open http://0.0.0.0:7860 in your browser. Features:

• Text input with Darija/Arabic
• Checkpoint selector (auto-discovers all .pth files)
• Adjustable speaker ID and pace
• Real-time audio playback

Accessing Demo Remotely

If the demo is running on a remote server, use SSH port forwarding to see it on your local browser:

# Run this on your LOCAL machine
ssh -L 7860:localhost:7860 user@remote-ip

Then open http://localhost:7860 on your local browser. Alternatively, use the --share flag to get a public URL:

bash models/tts/src/demo.sh models/tts/checkpoints --share

Training Features

Mixed Precision (AMP)

Enabled by default. Gives ~2x speedup and uses ~40% less VRAM.

# Disable if you encounter issues
VT_AMP=0 bash models/tts/src/finetune.sh

Learning Rate Scheduler

Cosine annealing with warm-up (1000 iters by default). Gradually decays the learning rate for better convergence.

Validation & Early Stopping

Automatically splits your dataset 90/10 and tracks validation loss. Saves best_model.pth when validation improves. Stops training if no improvement for 10 epochs.

Auto Checkpoint Cleanup

Automatically deletes old numbered checkpoints, keeping only the 3 most recent. Prevents disk-full crashes.

Environment Variables

All training features are controllable via environment variables:

Variable	Default	Description
VT_AMP	1	Enable mixed precision training
VT_KEEP_CKPTS	3	Number of checkpoints to keep (0 = disable cleanup)
VT_VAL_SPLIT	0.1	Validation split ratio (0 = disable validation)
VT_PATIENCE	10	Early stopping patience (0 = disable)
VT_WARMUP_ITERS	1000	LR warmup iterations
VT_LR_MIN_RATIO	0.01	Minimum LR as ratio of initial LR
VT_GAN_WARMUP_ITERS	1000	GAN loss warmup iterations
VT_RESUME_OPTIMIZERS	0	Resume optimizer states from checkpoint
VT_RESUME_PROGRESS	0	Resume epoch/iter counters from checkpoint
VT_DISABLE_TB	0	Disable TensorBoard logging

Example with custom settings:

VT_KEEP_CKPTS=5 VT_PATIENCE=20 VT_AMP=1 bash models/tts/src/finetune.sh

Export Artifacts (Important for Cloud Instances)

Before terminating your VM/instance, archive and download outputs:

cd /workspace/Voice_trainer
tar -czf export_voice_trainer_$(date +%Y%m%d_%H%M).tar.gz \
  models/tts/checkpoints \
  models/tts/src/config.yaml \
  models/tts/results \
  models/tts/logs

Then copy locally with scp:

scp -P <PORT> root@<IP>:/workspace/Voice_trainer/export_voice_trainer_*.tar.gz .

Troubleshooting

Hugging Face auth issues

• Use hf auth login (new CLI) in the same environment running setup.
• Confirm access to DODa on Hugging Face.

GPU detected but not used

• Verify with:

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'no-gpu')"

invalid phonemes messages

• Some rows may be skipped by tokenizer/phonemizer logic.
• A small amount is expected; too many may reduce final quality.

NaN/unstable training

• AMP handles most NaN/Inf gradient issues automatically now.
• If problems persist: VT_AMP=0 to disable mixed precision.
• Reduce learning rates in models/tts/src/config.yaml (g_lr, d_lr).

Disk space issues

• Auto-cleanup is now enabled by default (keeps 3 checkpoints).
• Adjust with VT_KEEP_CKPTS=5 for more safety margin.

Project Layout

Voice_trainer/
├── scripts/
│   ├── setup-deps.sh           ← Stage 1: install packages + weights
│   ├── download-dataset.sh     ← Stage 2: download + format DODa
│   ├── prepare-dataset.sh      ← Stage 3: merge, WAV, F0, config
│   ├── setup_tts_doda.sh       ← All-in-one (chains the 3 above)
│   └── download_format_doda.py
├── models/
│   └── tts/
│       ├── src/
│       │   ├── finetune.sh
│       │   ├── test-ckpts.sh
│       │   ├── demo.sh
│       │   ├── demo.py
│       │   ├── train_fp_adv.py
│       │   ├── generate-config.py
│       │   ├── extract_f0_penn.py
│       │   ├── download_files.py
│       │   └── test_raw_model.py
│       └── tts-arabic-pytorch/
├── tools/dataset/
│   ├── merge-datasets.py
│   └── mp3-to-wav.sh
├── requirements.txt
└── pyproject.toml

License

AGPL-3.0. See LICENSE.

Credits

• nipponjo/tts-arabic-pytorch
• atlasia/DODa-audio-dataset