Skip to content

Adjust tokenizer

Choose a tag to compare

View all tags
@FrandyEddy FrandyEddy released this 30 Jun 09:42
· 59 commits to master since this release
v0.1.10.1
5b19b40
  • Adjust tokenizer so that some tokens that is personal pronoun or particle will be split
Assets 2
Loading