Add component exports matching Python tokenizers library structure#10

Draft

Copilot wants to merge 10 commits intomainfrom

copilot/add-additional-exports

Commits on Nov 4, 2025

Initial plan
Copilot
committed
Add additional exports for Encoding, pre-tokenizers, and models

Copilot
and
xenova
committed
Add exports for normalizers, decoders, and post-processors

Copilot
and
xenova
committed
Add comprehensive tests for package exports and update Jest config

Copilot
and
xenova
committed
Update README with component export examples

Copilot
and
xenova
committed
Fix exports to match Python API: remove duplicates, add base classes, use correct short names

Copilot
and
xenova
committed
Use single bundle with tiny wrapper files for submodule exports

Copilot
and
xenova
committed
Simplify to single main export with unique names for all components

Copilot
and
xenova
committed
Remove unnecessary renaming in exports - use final names throughout

Copilot
and
xenova
committed
Cleanup
xenova
committed