|
How does memory scale with batch size and sequence length? |
Answered by
Ultron09
Aug 28, 2026
Replies: 1 comment
|
The memory scales as O(B * H * S^2) for the attention matrix and O(B * S * D) for activations. |
0 replies
Answer selected by
Ultron09
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The memory scales as O(B * H * S^2) for the attention matrix and O(B * S * D) for activations.