Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
contents		contents
data		data
scripts		scripts
.gitignore		.gitignore
README.md		README.md
generate_test.ipynb		generate_test.ipynb
run_hebo.py		run_hebo.py
test_run.sh		test_run.sh
train.py		train.py

Repository files navigation

Nano MDM

Decoding Process Visualization

trainer with DDP
generation (perplexity / diversity traderoff with reference model?)
Sweep learning rate
Sweep width and see muP happening

About

Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun

Custom properties

Report repository

Releases

No releases published

Packages

Contributors

Languages

Jupyter Notebook 99.7%
Other 0.3%