v1.2.1

   🐞 Bug Fixes

• Add underscore to a punctuation pattern  -  by @matej21 in #3 (19d69)

    View changes on GitHub

v1.2.0

   🚀 Features

• New splitByTokens function  -  by @johannschopplich (b9db6)

   🏎 Performance

• Improve tsdown dist build  -  by @johannschopplich (44b4a)
• Optimize tsdown build size  -  by @johannschopplich (28563)

    View changes on GitHub

v1.1.0

   🚀 Features

• Add a new sliceByTokens  -  by @alephpiece (85af4)

    View changes on GitHub

v1.0.1

No significant changes

    View changes on GitHub

v1.0.0

   🚀 Features

• Library overhaul for any LLM  -  by @johannschopplich (d5e20)

    View changes on GitHub

v0.4.1

   🚨 Breaking Changes

• Drop unused getEmbeddingContextSize function  -  by @johannschopplich (3ee97)

   🏎 Performance

• Cache split regex  -  by @johannschopplich (94331)

    View changes on GitHub

v0.4.0

   🚨 Breaking Changes

• Rename package to tokenx  -  by @johannschopplich (4a8ed)

    View changes on GitHub

v0.3.6

   🚀 Features

• Support GPT-4o "omni" model  -  by @johannschopplich (5e03f)

    View changes on GitHub

v0.3.5

   🚀 Features

• Update latest GPT model names  -  by @johannschopplich (2f6ff)

    View changes on GitHub

v0.3.4

   🐞 Bug Fixes

• Add missing gpt-4-turbo-preview  -  by @johannschopplich (a57b8)

    View changes on GitHub