Question

[fakerybakery]

Hi,
Thanks for releasing this code. Does this codebase decrease the size of the model (ie file size, required VRAM)?
Thank you!

[dkmisra]

This is one of the feature of LASER but currently the code doesn't do that. This is because we do SVD on a mxn W matrix which gives us U, S, V matrices of size m x m, mxn and nxn respectively, but then we take the top-k dimensions and multiply them back giving back a mxn low-rank approximation of W. To save memory, we should not multiple these matrices back but instead store them as 3 separate matrices of size mxk, kxk, kxn where k is the required low-rank.

The code needs to be modified so that instead of using the modified W parameter we trigger matrix multiplication with 3 separate matrices. This will cut down memory time albeit it will increase the number of sequential steps.

I am adding this as a feature request and we should be able to support this.

[dkmisra]

Related to #9