Learn how the attention mechanism works, from intuition to implementation.
Open Exercise 0 and follow along.
| Exercise | Description |
|---|---|
| Exercise 0 | The problem attention solves |
| Exercise 1 | Words as vectors |
| Exercise 2 | Dot product as similarity |
| Exercise 3 | The attention mechanism |
| Exercise 4 | Softmax: fixing the scores |
| Exercise 5 | The weighted sum |