We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent a7c08af commit 9b72679Copy full SHA for 9b72679
CHANGELOG.md
@@ -1,5 +1,13 @@
1
### 0.43.2
2
3
+This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.
4
+
5
+For each sequence (i.e. batch size increase of one) we expect memory savings of:
6
+- 405B: 39GB for seqlen 1024, and 4888GB for 128k
7
+- 70B: 20.1GB for 1024 and 2516GB for 128k
8
9
+This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
10
11
#### Improvements:
12
13
- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)
0 commit comments