Learn how to develop language model by developing tiny language models.
- Playing with language models by prompting
- Data preparation
- Tinystories taste synthetic data
- wikipedia deta
- Tokenizer
- Ngram language model
- Attention
- Pretraining
- Instruction finetuning
- RLHF (RLAIF)
# install pytorch following https://pytorch.org/
# other libraries
pip3 install -r requirements.txt