Skip to content

Commit 48cd9c3

Browse files
authored
Add Supertonic2 TTS support (#3094)
This pull request significantly enhances the offline Text-to-Speech capabilities of sherpa-onnx by integrating the Supertonic 2 TTS system. It introduces a complete pipeline for Supertonic 2, encompassing model configuration, a sophisticated multilingual text preprocessing module, and the core inference logic. The changes enable users to perform high-quality, on-device speech synthesis with support for multiple languages and advanced features like batch processing and customizable synthesis parameters, thereby broadening the application scope and improving the overall performance of the TTS module.
1 parent 9aa88f9 commit 48cd9c3

29 files changed

Lines changed: 4001 additions & 39 deletions
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
name: export-supertonic-to-int8-onnx
2+
3+
on:
4+
push:
5+
branches:
6+
- supertonic
7+
8+
workflow_dispatch:
9+
10+
concurrency:
11+
group: export-supertonic-to-int8-onnx-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
jobs:
15+
export-supertonic-to-onnx:
16+
if: github.repository_owner == 'k2-fsa' || github.repository_owner == 'csukuangfj'
17+
name: export supertonic int8
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- uses: actions/checkout@v4
22+
23+
- name: Setup Python
24+
uses: actions/setup-python@v5
25+
with:
26+
python-version: "3.10"
27+
28+
- name: Install Python dependencies
29+
shell: bash
30+
run: |
31+
pip install numpy onnx onnxruntime modelscope
32+
33+
- name: Run
34+
shell: bash
35+
run: |
36+
cd scripts/supertonic
37+
./run.sh
38+
39+
- name: Collect results
40+
shell: bash
41+
run: |
42+
src=scripts/supertonic
43+
d=supertonic-int8
44+
45+
mkdir $d
46+
cp -a LICENSE $d/LICENSE
47+
cp -a $src/README.md $d/README.md
48+
cp -v $src/onnx_int8/*.int8.onnx $d/
49+
[ -f $src/assets/onnx/unicode_indexer.bin ] && cp -v $src/assets/onnx/unicode_indexer.bin $d/
50+
[ -f $src/assets/onnx/tts.json ] && cp -v $src/assets/onnx/tts.json $d/
51+
[ -f $src/assets/voice_styles/voice.bin ] && cp -v $src/assets/voice_styles/voice.bin $d/voice.bin
52+
ls -lh $d/
53+
tar cjfv $d.tar.bz2 $d
54+
55+
ls -lh $d.tar.bz2
56+
57+
- name: Release
58+
if: github.repository_owner == 'csukuangfj'
59+
uses: svenstaro/upload-release-action@v2
60+
with:
61+
file_glob: true
62+
file: ./*.tar.bz2
63+
overwrite: true
64+
repo_name: k2-fsa/sherpa-onnx
65+
repo_token: ${{ secrets.UPLOAD_GH_SHERPA_ONNX_TOKEN }}
66+
tag: tts-models
67+
68+
- name: Release
69+
if: github.repository_owner == 'k2-fsa'
70+
uses: svenstaro/upload-release-action@v2
71+
with:
72+
file_glob: true
73+
file: ./*.tar.bz2
74+
overwrite: true
75+
tag: tts-models
76+
77+
- name: Publish to huggingface
78+
env:
79+
HF_TOKEN: ${{ secrets.HF_TOKEN }}
80+
uses: nick-fields/retry@v3
81+
with:
82+
max_attempts: 20
83+
timeout_seconds: 200
84+
shell: bash
85+
command: |
86+
git config --global user.email "csukuangfj@gmail.com"
87+
git config --global user.name "Fangjun Kuang"
88+
89+
export GIT_LFS_SKIP_SMUDGE=1
90+
export GIT_CLONE_PROTECTION_ACTIVE=false
91+
92+
d=supertonic-int8
93+
if [[ ! -d $d ]]; then
94+
echo "$d does not exist"
95+
exit 0
96+
fi
97+
98+
rm -rf huggingface
99+
git clone https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$d huggingface
100+
cd huggingface
101+
rm -rf ./*
102+
103+
git lfs track "*.onnx"
104+
cp -a ../$d/* ./
105+
106+
git add .
107+
ls -lh
108+
git status
109+
git commit -m "add supertonic int8 models" || true
110+
git push https://csukuangfj:$HF_TOKEN@huggingface.co/csukuangfj/$d main || true

scripts/supertonic/README.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Supertonic TTS INT8 Quantization
2+
3+
Quantize [Supertonic](https://github.com/supertone-inc/supertonic) TTS ONNX models to INT8 for on-device deployment.
4+
5+
## Overview
6+
7+
- **Pipeline**: `gen_calib_configs``dump_inputs``convert`; stage 4 generates **.bin** assets when JSONs exist: `generate_voices_bin.py`, `generate_indexer_bin.py`. Runtime loads **tts.json** for TTS config.
8+
- **Quantization**: duration_predictor, text_encoder, vector_estimator → dynamic INT8; vocoder → static INT8 (calibration from dumped data).
9+
- **Voice**: Runtime loads one **`voice.bin`**. Generate with `python3 generate_voices_bin.py [input_dir] [output_bin]`. Pass `--supertonic-voice-style=/path/to/voice.bin`. Use `--sid` 0..N-1 to select speaker.
10+
- **Unicode indexer**: Runtime uses **`unicode_indexer.bin`**. Generate with `python3 generate_indexer_bin.py [json_path] [bin_path]`. Pass `--supertonic-unicode-indexer=/path/to/unicode_indexer.bin`.
11+
- **TTS config**: Runtime loads **`tts.json`**. Pass `--supertonic-tts-json=/path/to/tts.json`.
12+
13+
## Usage
14+
15+
```bash
16+
./run.sh # Run all stages (0–4)
17+
./run.sh 4 # Only generate voice.bin, unicode_indexer.bin
18+
```
19+
20+
**Stages:** 0 = download models, 1 = gen calib configs, 2 = dump calib data, 3 = quantize, 4 = generate `voice.bin`, `unicode_indexer.bin`.

0 commit comments

Comments
 (0)