0.2.0

• Use FlashAttention
• Use SentencePiece BPE tokenizer
• Increased context length
• Added gradient checkpointing
• Support combining multiple datasets
• Download pre-trained tokenizer
• Optional encoding on the fly
• Added greedy search in inference
• Support continued prompts in inference (not the start of the sequence)
• Compute dataset and vocabulary statistics
• Data validation

0.1.2

• Documentation improvements

This release does not include updated weights. Please use checkpoints from previous releases.

0.1.1

• Added documentation
• Distribute a PyPI package
• Inference fixes

0.1.0

Initial release