Skip to content

Commit 27a66dd

Browse files
committed
fix: chars blank
1 parent 7606d94 commit 27a66dd

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

tensorflow_asr/tokenizers.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,8 @@ def __init__(self, decoder_config: DecoderConfig):
199199
line = unicodedata.normalize(self.decoder_config.normalization_form, line.lower()).strip("\n")
200200
if line.startswith("#") or not line:
201201
continue
202+
if line == "<blank>":
203+
line = ""
202204
self.tokens.append(line)
203205
if self.blank is None:
204206
self.blank = len(self.tokens) # blank not at zero

0 commit comments

Comments
 (0)