Neural Networks: Zero to Hero

This repository contains my re-implementations of the code in Andrej Karpathy's Neural Networks: Zero to Hero series.

• gpt2_repro: replicating GPT training almost exactly as described in the GPT-2/GPT-3 papers
• makemore: character-level language model that "makes more of the input" (implemented using various models)
• micrograd: scalar-level autograd engine
• minbpe: BPE tokenizer that supports training and inference
• nanogpt: similar to makemore, but uses a GPT-like architecture (stacked decoders)