43 lines (24 loc) · 1.29 KB

Helpful links (WIP)

Math behind DL

Linear algebra visualization: https://textbooks.math.gatech.edu/ila/index.html

Model architectures

Transformer visualization: https://poloclub.github.io/transformer-explainer/

Implementing LLM from scratch: https://github.com/rasbt/LLMs-from-scratch

LLM agents

UC Berkeley CS294/194-196 Large Language Model Agents: https://rdi.berkeley.edu/llm-agents/f24

Webshop (LLM Agent): https://webshop-pnlp.github.io/

MoE

Open-source MOE: https://github.com/XueFuzhao/OpenMoE

Miscellaneous

Ahead of AI: https://magazine.sebastianraschka.com/

Conference on language modeling: https://colmweb.org/index.html

LLM interpretability

Sparse autoencoders

https://github.com/KexinGAO42/Large-Language-Model-Knowledge-Base/tree/main/llm_interpretability

Monosemanticity (Anthropic)

Extracting interpretable features from a one-layer transformer by using a sparse autoencoder: https://transformer-circuits.pub/2023/ monosemantic-features/index.html Claude 3 Sonnet example: https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html

Circuit tracing (Anthropic)

Method details: https://transformer-circuits.pub/2025/attribution-graphs/methods.html

Claude 3.5 Haiku examples: https://transformer-circuits.pub/2025/attribution-graphs/biology.html