Commit 81b917d
authored
Possible typo in week1-01-attention (#60)
* Possible typo in week1-01-attention
Hello, was going through the book! I'm not 100% sure of this, but after going through the tests for day1-task2, it looks like the w_qkv matrices and w_o matrix have their shape reversed.
I confirmed by checking the mlx.nn.layers.linear.Linear weight, which is of shape `[Output, Input]`. Since w_qkv's output is HxD and input is E, the shape should be `[H x D, E]`.
* Oops fix another typo1 parent 04149a3 commit 81b917d
1 file changed
+2
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
118 | 118 | | |
119 | 119 | | |
120 | 120 | | |
121 | | - | |
| 121 | + | |
122 | 122 | | |
123 | | - | |
| 123 | + | |
124 | 124 | | |
125 | 125 | | |
126 | 126 | | |
| |||
0 commit comments