Home

Welcome to the Monarch-LLM wiki!

CURRENT CODEBASE:

import torch from mamba_ssm import Mamba2

batch, length, dim = 2, 64, 16 x = torch.randn (batch, length, dim).to("cuda")

model = Mamba2 ( d_model = dim, d_state = 64 d_conv = 4, expand = 2, ).to("cuda")

y = model assert y.shape == x.shape