feat(model): add Gemma-4 E4B support (layer spec, checkpoint loader, parity check)#4148

Open

DOGEUNNKIM wants to merge 28 commits into

NVIDIA-NeMo:mainfrom

DOGEUNNKIM:main

Commits on Jun 4, 2026

feat(model): add Gemma4 layer spec with dual-RoPE, PLE, and shared-KV

DOGEUNNKIM
and
claude
committed
feat(ckpt): add Gemma4 HF-to-Megatron checkpoint loader

DOGEUNNKIM
and
claude
committed
feat(example): add Gemma4 E4B parity check and training scripts

DOGEUNNKIM
and
claude
committed
docs(gemma4): add Gemma4 E4B usage guide

DOGEUNNKIM
and
claude
committed
feat(model): add Gemma4E4BProvider for clean-MCore compatibility

DOGEUNNKIM
and
claude
committed
Merge branch 'main' into main
DOGEUNNKIM
authored
docs(gemma4): update README to reflect Bridge-based architecture

DOGEUNNKIM
and
claude
committed
refactor(example): rename train_gemma4_e4b_pipeline.sh to slurm_pretrain.sh

DOGEUNNKIM
and
claude
committed
Edit gemma4_bridge.py to follow file struct conventions
DOGEUNNKIM
committed
ADD gemma dense omni mdality model
DOGEUNNKIM
committed

Commits on Jun 5, 2026

Commits on Jun 6, 2026

Add Gemma4 dense model recipe test
DOGEUNNKIM
committed

Commits on Jun 9, 2026

Commits on Jun 12, 2026

Merge branch 'main' into main
DOGEUNNKIM
authored

Commits on Jun 16, 2026