Add block sparse linear and locally connected layers#6

Draft

clane9 wants to merge 9 commits into

mainfrom

Commits on Jul 1, 2024

Initial blocksparse linear outline
clane9
committed
BlockSparseLinear complete

alismil
authored and
clane9
committed
added MixtureLayerNorm

alismil
authored and
clane9
committed
matched MixtureLinear to main

alismil
authored and
clane9
committed
completed BlockSparseLocallyConnected

alismil
authored and
clane9
committed
updated comments in _create_connectivity_matrix

alismil
authored and
clane9
committed

Commits on Jul 2, 2024

Update blocksparse linear implementation
clane9
committed

Commits on Jul 3, 2024