Skip to content

Commit 9b72679

Browse files
Changelog: add explanation r. QLoRA mem savings
1 parent a7c08af commit 9b72679

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

CHANGELOG.md

+8
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
### 0.43.2
22

3+
This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.
4+
5+
For each sequence (i.e. batch size increase of one) we expect memory savings of:
6+
- 405B: 39GB for seqlen 1024, and 4888GB for 128k
7+
- 70B: 20.1GB for 1024 and 2516GB for 128k
8+
9+
This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.
10+
311
#### Improvements:
412

513
- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)

0 commit comments

Comments
 (0)