Skip to content

Commit 7800734

Browse files
Changelog: add explanation r. QLoRA mem savings
1 parent a7c08af commit 7800734

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

CHANGELOG.md

+8
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
### 0.43.2
22

3+
This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.
4+
5+
For each sequence (i.e. batch size increase of one) we expect memory savings of:
6+
- 405B: 39GB for `seqlen=1024`, and 4888GB for `seqlen=128,00`
7+
- 70B: 10.1GB for `seqlen=1024` and 1258GB for `seqlen=128,00`
8+
9+
This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
10+
311
#### Improvements:
412

513
- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)

0 commit comments

Comments
 (0)