File tree 1 file changed +22
-0
lines changed
1 file changed +22
-0
lines changed Original file line number Diff line number Diff line change @@ -18,6 +18,28 @@ You may join skyzh's Discord server and study with the tiny-llm community.
18
18
19
19
[ ![ Join skyzh's Discord Server] ( book/src/discord-badge.svg )] ( https://skyzh.dev/join/discord )
20
20
21
+ ## Roadmap
22
+
23
+ | Week + Chapter | Topic | Code | Test | Doc |
24
+ | -------------- | ----------------------------------------------------------- | ---- | ---- | --- |
25
+ | 1.1 | Attention | ✅ | ✅ | ✅ |
26
+ | 1.2 | RoPE | ✅ | ✅ | ✅ |
27
+ | 1.3 | Grouped Query Attention | ✅ | 🚧 | 🚧 |
28
+ | 1.4 | RMSNorm and MLP | ✅ | 🚧 | 🚧 |
29
+ | 1.5 | Transformer Block | ✅ | 🚧 | 🚧 |
30
+ | 1.6 | Load the Model | ✅ | 🚧 | 🚧 |
31
+ | 1.7 | Generate Responses | ✅ | ✅ | 🚧 |
32
+ | 2.1 | KV Cache | 🚧 | 🚧 | 🚧 |
33
+ | 2.2 | Quantized Matmul and Linear (CPU) | 🚧 | 🚧 | 🚧 |
34
+ | 2.3 | Quantized Matmul and Linear (Metal) | 🚧 | 🚧 | 🚧 |
35
+ | 2.4 | Attention Kernel | 🚧 | 🚧 | 🚧 |
36
+ | 2.5 | Softmax Kernel | 🚧 | 🚧 | 🚧 |
37
+ | 2.6 | Prompt Cache / Multiple Requests | 🚧 | 🚧 | 🚧 |
38
+ | 2.7 | Benchmarking | 🚧 | 🚧 | 🚧 |
39
+ | 3.1 | API Server | 🚧 | 🚧 | 🚧 |
40
+ | 3.2 | Continuous Batching | 🚧 | 🚧 | 🚧 |
41
+
42
+
21
43
<!--
22
44
23
45
### Day 2: RoPE Embedding
You can’t perform that action at this time.
0 commit comments