Add suppport for Thai - #117

Open

jadechip wants to merge 30 commits into

myshell-ai:mainfrom

jadechip:add-thai

Commits on Apr 29, 2024

Add suppport for Thai

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
ffd8f41
Add Thai Wiktionary Corpus

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
c960e81
Fix issues with g2p function

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
80460cc
Add better test cases

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
4d1fd9c
Remove debug line

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
6d7f2ac
More fixes

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
aff4b04
Fix normalize and dictionary

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
5cd7da1
Update expected format of test_word_tokenize test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
4484651

Commits on Apr 30, 2024

Fix format of test_thai_text_to_phonemes test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
17cafba
Add test to compare with Korean

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
1c74eb7
Clean up import statements

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
11c5a75

Commits on May 8, 2024

Add missing code for training Thai language and Thai Grapheme to Phoneme corpus file

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
d8a5e13

Commits on May 9, 2024

Initialize the TextEncoder with fixed size

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
28dd03e
Add get_resized_embeddings to SynthesizerTrn class

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
809a043
Use self.enc_p.emb

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
818e4b4
Remove th_symbols from sorted list

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
b25a99d

Commits on May 10, 2024

Fix unsupported characters in g2p function

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
11c55ef

Commits on May 16, 2024

Fix tones list from g2p function being initliazed to zeroes and adjust test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
3004182
Assign multiple phones based on the number of phones

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
5b1faee
Fix regression

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
c1870d9

Commits on May 18, 2024

Fix tones mapping

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
adb9232
Add new test case

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
17a3bd7

Commits on May 19, 2024

Change format of word2ph to inlclude count

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
5c19c03

Commits on May 20, 2024

Keep special characters in word2ph to be consistent with other languages

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
a65cc2f
Update test case for Thai bert to match format without special cases

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
af38d49

Commits on May 28, 2024

Fix dictionary lookup and use bert tokenizer in g2p

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
2be87f2
Fix tone issues and add warning for mismatch due to _ underscore character

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
120bcfd
Squash 2 nasty bugs: 1.) Assign tone value to each phoneme excluding the tone marker only if there is one and Keep only non-empty groups for cases with a trailing dot

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
1251906

Commits on Jun 5, 2024

Add gradient clipping, process underscores to align with bert features, add additional symbols from error during training and update tones

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
d4c9124

Commits on Aug 7, 2024

Refactor Thai g2p lookup and add new tokenizer

Jim Eric Skogman
authored and
Jim Eric Skogman
committed
769dbf8