[levanter] Add generation_config.json support for chat model checkpoints#4160

Merged

ahmeda14960 merged 15 commits intomainfrom

instruct_tokenizer

Mar 26, 2026

Commits on Mar 26, 2026

Add generation_config.json support for chat model HF checkpoints

ahmeda14960
and
claude
committed
Move build_generation_config into hf_checkpoints.py

ahmeda14960
and
claude
committed
Document hf_generation_eos_token_ids in DPO training guides

ahmeda14960
and
claude
committed
Add smoke test for generation_config.json in DPO checkpoints

ahmeda14960
and
claude
committed
Fix smoke test: use ExecutorStep pattern for tokenized data

ahmeda14960
and
claude
committed
Switch smoke test to v5p-16 (available in us-east5-a)

ahmeda14960
and
claude
committed
Switch smoke test to v5e-4 (idle workers available)

ahmeda14960
and
claude
committed
Switch smoke test to v5e-8 (smallest valid v5e slice)

ahmeda14960
and
claude
committed
Fix TPU type: v5litepod-8 (not v5e-8)

ahmeda14960
and
claude
committed
Switch to v5p-8 in us-central1 (8 available, same region as CPU executor)

ahmeda14960
and
claude
committed
Bump seq_len to 2048 (some ultrafeedback examples exceed 512)

ahmeda14960
and
claude
committed
Bump to seq_len=4096 (full Llama 3 context, some UF examples >2048)

ahmeda14960
and
claude
committed
Set ram=400g for DPO smoke test (OOM during HF save at 128g default)

ahmeda14960
and
claude
committed
Add LLAMA3_CHAT_STOP_TOKEN_IDS to dpo_ultrafeedback experiment

ahmeda14960
and
claude
committed
Merge branch 'main' into instruct_tokenizer
ahmeda14960
authored