Skip to content

Commit 8f90f90

Browse files
committed
Revert "fix: Use non-traditional RoPE in Qwen2 test case. (#56)"
This reverts commit bf3383d.
1 parent 81b917d commit 8f90f90

File tree

2 files changed

+1
-3
lines changed

2 files changed

+1
-3
lines changed

book/src/week1-03-gqa.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,8 +108,6 @@ x = scaled_dot_product_attention_grouped(q, k, v, scale, mask) -> B, L, H_q, D ;
108108
x = linear(x, wo) -> B, L, E
109109
```
110110

111-
Keep in mind that you should use non-traditional RoPE.
112-
113111
You can test your implementation by running the following command:
114112

115113
```bash

tests_refsol/test_week_1_day_3.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ def test_task_3_qwen2_grouped_query_attention(
158158
rms_norm_eps=1e-6,
159159
vocab_size=1000,
160160
rope_theta=theta,
161-
rope_traditional=True,
161+
rope_traditional=False,
162162
max_position_embeddings=max_seq_len,
163163
)
164164

0 commit comments

Comments
 (0)