Hi, using xformers monkey patch training llama2 got loss explosion #35
Description
By using xformers to train llama2, the loss are explosion, do u know why? On V100 only
{{ message }}
By using xformers to train llama2, the loss are explosion, do u know why? On V100 only