"Toy Models of Superposition" replication

This is a replication of the Anthropic paper "Toy Models of Superposition" by Elhage et al. (2022). Best viewed on Colab:

• Demonstrating superposition
• Superposition as a phase transition

Demonstrating superposition

Here's my plot:

Here's Anthropic's plot:

Superposition as a phase transition

Here's my plot:

Here's Anthropic's plot:

---

To do:

• (Tentative) Replicate the sections on feature geometry and computation in superposition.