Need to try power-law compression loss

[seungwonpark]

When I was implementing this repository, I didn't know what power-law compression means in the original paper.
Thanks to Quan Wang's tutorial video, now I can understand it. Now I need to try that loss instead of using MSE loss.
https://www.youtube.com/watch?v=gnRX2lzepz0

[stegben]

I could help with this if you haven't started yet!

[seungwonpark]

@stegben That will be very helpful. Thanks a lot!
Actually, I don't have spare time/resource to try this now. So whenever you get a new (perhaps better) result with this, let's discuss that.

Plus, whenever you try a new thing, feel free to make a pull request about that.

[Andrew-Infinite]

Even though it might not be completely relevant, I will just share my current finding here, this is what I had tested.
I am training a (hopefully correct) causal network with only 5 layer CNN with a 3 layer unidirectional LSTM.

MSE came out to be better than (magnitude only) power law. I think the reason is because, MSE and magnitude only power law were proportional just that it consistently give me a smaller loss number, therefore a "slower" learning, thus worse SDR result

I haven't tried complex power law, the complex term would most probably give it a different path compare to MSE and magnitude only power law.