PROJECT_CHECKLIST.md

Project checklist

Phase 1 — Attention with numbers

• Calculate a 3-token, 2-dimensional attention example by hand
• Apply scaling and a causal mask
• Check every softmax row
• Repeat with two heads and concatenate the outputs

Phase 2 — Components

• Token embeddings
• Position embeddings
• Q/K/V projections
• Causal multi-head attention
• Feed-forward layer
• Residual paths
• Pre-norm and post-norm variants

Phase 3 — Training

• Overfit one short sequence
• Verify shifted next-token targets
• Train the baseline
• Save config, vocabulary, seed, logs, and checkpoint
• Generate with a fixed prompt and seed

Phase 4 — Comparison

• Match parameter counts across norm variants
• Run at least three seeds if the budget permits
• Plot train and validation loss
• Inspect gradient norms

Phase 5 — Publication

• Add a tensor-shape diagram or table
• Replace README placeholders with measured values
• Finish BLOG_DRAFT.md
• Link code, article, and portfolio