Please consider adding MIG (MI-rror with G-radient modification)

[YagaoDirac]

Feature description

A engineeringly better implementation of FCNN( torch.nn.Linear)

Feature motivation

(Optional) Suggest a Solution

(pytorch/pytorch#122680)
The code is now implemented in pytorch.
If you are interested in the correspoding paper, sorry, no paper at the moment. I explained how it works in code.
I'm now having fun learning rust. Maybe I will write a rust version for MIG probably in Burn.

[antimora]

You are welcome to submit a PR with specifics. So you know the linear module is very simple that relies on matmul operation. So your mentioned optimization is at this level (backend). There is a lot to be done there. So I recommend you look into this.

I do agree Rust is a lot of fun. Happy coding and learning!

[antimora]

I moved this issue to the discussion board because we cannot specific action on this. At the moment it's broad and ambiguous.

[YagaoDirac]

Thanks. I'm still fighting against Rust grammar. I'll come back later.