Add MoeAdamHHeuristic, drop dense layers, fix align_kv_heads sharding#4636

Open

ClassicLarry wants to merge 8 commits intomainfrom

grug_moe_heuristic

Commits on Apr 10, 2026

Add MoeAdamHHeuristic, drop dense layers, fix align_kv_heads sharding
ClassicLarry
committed
Add isoflop curvature stability criterion to promotion gates
ClassicLarry
committed
Drop unused seq_len kwarg from heuristic helpers
ClassicLarry
committed
Expose heuristic baseline as GRUG_MOE_TRIAL_MODEL for canary_ferry
ClassicLarry
committed

Commits on Apr 12, 2026

Fix layer counts and wandb links in moe/README.md

ClassicLarry
and
claude
committed

Commits on Apr 13, 2026

Merge branch 'main' into grug_moe_heuristic
ClassicLarry
authored

Commits on Apr 15, 2026