We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
1 parent a7c08af commit 7800734Copy full SHA for 7800734
CHANGELOG.md
@@ -1,5 +1,13 @@
1
### 0.43.2
2
3
+This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.
4
+
5
+For each sequence (i.e. batch size increase of one) we expect memory savings of:
6
+- 405B: 39GB for `seqlen=1024`, and 4888GB for `seqlen=128,00`
7
+- 70B: 10.1GB for `seqlen=1024` and 1258GB for `seqlen=128,00`
8
9
+This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
10
11
#### Improvements:
12
13
- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)
0 commit comments