Skip to content

Commit dbaf894

Browse files
authored
Add JavaScript (WebAssembly) example for ZipVoice TTS (#3341)
1 parent cc02b69 commit dbaf894

3 files changed

Lines changed: 81 additions & 0 deletions

File tree

.github/scripts/test-nodejs-npm.sh

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,17 @@ rm -fv enhanced*.wav
157157

158158
# offline tts
159159

160+
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
161+
tar xf sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
162+
rm sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
163+
164+
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/vocoder-models/vocos_24khz.onnx
165+
166+
node ./test-offline-tts-zipvoice-zh-en.js
167+
ls -lh *.wav
168+
rm -rf sherpa-onnx-zipvoice-distill-int8-zh-en-emilia
169+
rm -f vocos_24khz.onnx
170+
160171
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-multi-lang-v1_0.tar.bz2
161172
tar xf kokoro-multi-lang-v1_0.tar.bz2
162173
rm kokoro-multi-lang-v1_0.tar.bz2

nodejs-examples/README.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,24 @@ node ./test-offline-speaker-diarization.js
8181

8282
In the following, we demonstrate how to run text-to-speech.
8383

84+
## ./test-offline-tts-zipvoice-zh-en.js
85+
86+
[./test-offline-tts-zipvoice-zh-en.js](./test-offline-tts-zipvoice-zh-en.js)
87+
shows how to use ZipVoice for Zero-shot TTS in Chinese and English.
88+
89+
Please make sure that the reference transcript matches the reference audio.
90+
91+
You can use the following command to run it:
92+
```bash
93+
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
94+
tar xf sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
95+
rm sherpa-onnx-zipvoice-distill-int8-zh-en-emilia.tar.bz2
96+
97+
curl -SL -O https://github.com/k2-fsa/sherpa-onnx/releases/download/vocoder-models/vocos_24khz.onnx
98+
99+
node ./test-offline-tts-zipvoice-zh-en.js
100+
```
101+
84102
## ./test-offline-tts-pocket-en.js
85103

86104
[./test-offline-tts-pocket-en.js](./test-offline-tts-pocket-en.js)
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
// Copyright (c) 2026 Xiaomi Corporation (authors: Fangjun Kuang)
2+
3+
const sherpa_onnx = require('sherpa-onnx');
4+
5+
function createOfflineTts() {
6+
const zipvoice = {
7+
encoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/encoder.int8.onnx',
8+
decoder: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/decoder.int8.onnx',
9+
tokens: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/tokens.txt',
10+
lexicon: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/lexicon.txt',
11+
dataDir: './sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/espeak-ng-data',
12+
vocoder: './vocos_24khz.onnx',
13+
};
14+
15+
const offlineTtsModelConfig = {
16+
offlineTtsZipVoiceModelConfig: zipvoice,
17+
numThreads: 1,
18+
debug: 1, // set it to 1 to see verbose logs; 0 to disable logs
19+
provider: 'cpu',
20+
};
21+
22+
const offlineTtsConfig = {
23+
offlineTtsModelConfig: offlineTtsModelConfig,
24+
maxNumSentences: 1,
25+
};
26+
27+
return sherpa_onnx.createOfflineTts(offlineTtsConfig);
28+
}
29+
30+
const referenceWaveFilename =
31+
'./sherpa-onnx-zipvoice-distill-int8-zh-en-emilia/test_wavs/leijun-1.wav';
32+
const wave = sherpa_onnx.readWave(referenceWaveFilename);
33+
34+
const referenceText =
35+
'那还是三十六年前, 一九八七年. 我呢考上了武汉大学的计算机系.';
36+
const text =
37+
'小米的价值观是真诚, 热爱. 真诚,就是不欺人也不自欺. 热爱, 就是全心投入并享受其中.';
38+
39+
const generationConfig = {
40+
referenceAudio: wave.samples,
41+
referenceSampleRate: wave.sampleRate,
42+
// It must match the transcript of the reference audio above.
43+
referenceText: referenceText,
44+
numSteps: 4,
45+
extra: {min_char_in_sentence: 10},
46+
};
47+
48+
const tts = createOfflineTts();
49+
const audio = tts.generateWithConfig(text, generationConfig);
50+
tts.save('./test-zipvoice-zh-en.wav', audio);
51+
console.log('Saved to test-zipvoice-zh-en.wav successfully.');
52+
tts.free();

0 commit comments

Comments
 (0)