When can I use the logsumexp I get from return_attn_probs?

[ad8e]

flash-attention/hopper/flash_attn_interface.py

Lines 848 to 850 in d234051

	return_attn_probs: bool. Whether to return the attention probabilities. This option is for
	testing only. The returned probabilities are not guaranteed to be correct
	(they might not have the right scaling).

@nshepperd traced the logic and found that the LSE is in log2 format. Is there anything else I should be aware of to use the LSE with FA3?

[tridao]

I don't think the LSE is in base 2, it should be in base e.