This repository is a place where I experiment and learn how transformers work.
I'm following along with resources from Andrej Karpathy's lectures to build my understanding of transformer architectures, attention mechanisms, and how these powerful models function under the hood.
- Understand the fundamental concepts behind transformers
- Implement transformer components from scratch
- Experiment with different architectures and approaches
- Document my learning journey
- Andrej Karpathy's educational content and lectures
- Hands-on coding experiments and implementations
This is a learning repository - expect experimental code and gradual improvements as I deepen my understanding of transformers.