[Agentic SFT] Post-train Marin-32B on the TerminalCorpus

[AlienKevin]

Description

Post-train marin-community/marin-32b-base on 15% of Nemotron-Terminal-Corpus with the same SFT recipe used in #4307, then evaluate on Terminal-Bench 2.0 against the Qwen3-32B SFT checkpoint at the same training fraction.

Hypothesis or Goal

Marin-32B base should absorb the same terminal-skills mix with comparable or better sample efficiency than the Qwen3-32B baseline when both are trained on 15% of the full corpus.

Links

• WandB Report:
• Issue [Agentic SFT] Reproduce NemotronTerminal-32B #4307: [Agentic SFT] Reproduce NemotronTerminal-32B #4307
• Marin 32B base: https://huggingface.co/marin-community/marin-32b-base

Results

Summary

This experiment fine-tuned marin-community/marin-32b-base for 859 steps on 15% of Nemotron-Terminal-Corpus and evaluated the checkpoint on Terminal-Bench 2.0 with the Harbor/Terminus-2 setup. Early failures were traced to an EOS metadata mismatch in the saved checkpoint, not to the SFT weights themselves; after patching the checkpoint and adding pipeline safeguards, the apples-to-apples eval still rejected the hypothesis. At a comparable training budget, Marin-32B solved 2/87 tasks (~2.3%) versus Qwen3-32B SFT's 18/86 (~20.9%), and Marin's two solves were also solved by Qwen3 at half its training budget. The current recommendation is to treat the 15% Marin-32B SFT as a negative result and only continue if testing full-corpus Marin SFT is worth the cost.

Helpful links

• Final closing result: [Agentic SFT] Post-train Marin-32B on the TerminalCorpus #4760 (comment)
• EOS root cause and pipeline fix: [Agentic SFT] Post-train Marin-32B on the TerminalCorpus #4760 (comment)
• Apples-to-apples post-fix eval: [Agentic SFT] Post-train Marin-32B on the TerminalCorpus #4760 (comment)
• Raw TB2 trajectories: https://huggingface.co/datasets/AlienKevin/terminal-bench-2-sft-traces

[AlienKevin]

🤖 Status update: Training in progress

Experiment script

Commit: c3bd727 on branch entire/a8b29ea-2ac78f

• experiments/exp4760_sft_marin_32b_base_terminal_corpus.py
• tests/test_nemotron_terminal_reproduction.py (test coverage)
• scripts/convert_qwen3_32b_checkpoint_to_hf.py (HF conversion)

Training config

Parameter	Value
Base model	marin-community/marin-32b-base
Tokenizer	marin-community/marin-tokenizer
Data	15% of Terminal-Corpus (4 subsets, ~54.9K effective examples)
Epochs	2
Batch size	128
Total steps	859
Seq len	32,768
LR	2e-5 (cosine, 10% warmup)
TPU	v5p-64 (preemptible, us-east5-a)
Tensor parallel	1

Launch command

uv run iris --config lib/iris/examples/marin.yaml job run \
    --tpu v5p-64 \
    --zone us-east5-a \
    -e MARIN_PREFIX gs://marin-us-east5 \
    -e WANDB_ENTITY marin-community \
    -e WANDB_PROJECT marin \
    -e WANDB_API_KEY ${WANDB_API_KEY} \
    -e HF_TOKEN ${HF_TOKEN} \
    -e TPU_CI true \
    -e TPU_VARIANT v5p-64 \
    --no-wait \
    -- python experiments/exp4760_sft_marin_32b_base_terminal_corpus.py

Current progress

• Iris job: /kevin/iris-run-exp4760_sft_marin_32b_base_terminal_corpus-20260416-152055/train_lm
• State: Running (8/8 tasks assigned on v5p-64)
• Training: Step 4 / 859 (checkpoint saved at step-4)
• Checkpoints: gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/
• Checkpoint interval: Every 214 steps

Note: Job was preempted 3 times before this successful launch. Tokenized datasets are cached in gs://marin-us-east5/tokenized/ and reused across restarts. Levanter will resume from the latest checkpoint if preempted again.

[AlienKevin]

🤖 Status update: Training in progress (step 46/859)

Progress

Training is running stably on v5p-64 in us-east5-a after resolving two issues:

1. OOM fix: Bumped host RAM from 128 GB to 256 GB — the 32B model at 32K seq len was being OOM-killed by the kernel around step 5-6
2. Checkpoint fix: Changed steps_per_checkpoint from 214 to 25 to survive preemption on preemptible TPUs

Metric	Value
Step	46 / 859 (5.4%)
Loss	1.01 (down from 3.37)
Rate	~62-82 s/step (after XLA cache warmup)
ETA	~14-18 hours from now
Preemptions	0 (this run)
Uptime	~4 hours

Checkpoints

Saved at: gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/checkpoints/

Latest: step-45

Iris job

/kevin/iris-run-exp4760_sft_marin_32b_base_terminal_corpus-20260418-023408/train_lm

Notes

• Wandb is running in offline mode (no live dashboard URL). Run data is saved locally on the worker.
• Prior to these fixes, the job was preempted ~120+ times and OOM-killed repeatedly without making progress past step 5.

[AlienKevin]

🤖 Wandb link

Training loss curve: https://wandb.ai/marin-community/marin/runs/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003

Current progress: step 210/859 (24.4%), loss ~0.56, MFU ~46.6%. Training on preemptible v5p-64, resuming from checkpoints after preemptions.

[AlienKevin]

🤖 Progress update: 73% complete

Training is at step 628/859 (73.1%), loss ~0.51. Running on preemptible v5p-64 in us-east5-a with automatic checkpoint resumption after preemptions.

Loss curve (from wandb)

Step	Loss	Progress
47	0.92	5.5%
118	0.62	13.7%
206	0.57	24.0%
294	0.54	34.2%
380	0.53	44.2%
488	0.51	56.8%
607	0.53	70.7%

Loss plateaued around 0.51-0.53 since step ~400 (epoch 2 territory).

Status

• Checkpoint: step-628 at gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/
• Remaining: 231 steps (~27%)
• Wandb: https://wandb.ai/marin-community/marin/runs/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003
• Preemption strategy: checkpointing every ~4 steps, auto-resubmit on preemption
• Estimated completion: within 12-24h depending on preemption frequency

[AlienKevin]

🤖 Training COMPLETE

The Marin-32B Terminal-Corpus 15% SFT has finished all 859 steps.

Final results

Metric	Value
Steps	859/859 (100%)
Final loss	~0.49
Epochs	2
Base model	marin-community/marin-32b-base
Data	15% Terminal-Corpus (54.9K examples)

Artifacts

• HF checkpoint: gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/hf/step-858/
• Levanter checkpoints: gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/checkpoints/
• Wandb: https://wandb.ai/marin-community/marin/runs/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003

Loss curve

Step	Loss
3	3.34
47	0.92
140	0.59
294	0.54
488	0.51
607	0.53
858	~0.49

Notes

• Trained on preemptible v5p-64 in us-east5-a with aggressive checkpointing (every ~4 steps)
• Required 256 GB host RAM (128 GB caused OOM at step 5-6)
• Survived numerous preemptions through checkpoint resumption

[AlienKevin]

🤖 TB2 Eval Update: Marin-32B SFT blocked on TPU vLLM

The Marin-32B SFT model (exp4760, step-858) cannot be evaluated on Terminal-Bench 2.0 using vLLM on TPU. The vLLM TPU backend hangs during the first inference request — the model's non-standard architecture is incompatible:

Config	Marin-32B	Qwen3-32B (works)
num_attention_heads	40	64
vocab_size	128256	151936
intermediate_size	27648	25600
rope_scaling	llama3	none

The Qwen3-32B SFT (exp4307) works fine on TPU vLLM and achieved 12/83 = 14.5% on TB2.

Next step: The Marin-32B model needs GPU-based vLLM evaluation (e.g., on H100s) to run TB2.

[AlienKevin]

🤖 TB2 Eval Results: Marin-32B SFT (3 trials)

Full Terminal-Bench 2.0 evaluation of exp4760-marin32b-sft-step858 using Harbor with Terminus-2 agent on v5p-8 TPUs, run 3 times.

Results

Trial	Tasks completed	Solved	Solve rate
Trial 1	89/89	1	1.1%
Trial 2	55/89	0	0.0%
Trial 3	18/89	0	0.0%
Combined	89/89	1	1.1%

Solved task: largest-eigenval (trial 1 only)

Comparison with Qwen3-32B SFT

Model	TB2 solve rate	Training data	Training progress
Marin-32B SFT (exp4760)	1/89 = 1.1%	15% Terminal-Corpus, 2 epochs	859/859 steps (100%)
Qwen3-32B SFT (exp4307)	15/87 = 17.2%	100% Terminal-Corpus, 2 epochs	500/5721 steps (8.7%)
Paper target	27.4%	100% Terminal-Corpus, full	Full training

Key observations

1. Marin-32B significantly underperforms Qwen3-32B (1.1% vs 17.2%) despite training to completion on its data fraction. This suggests the Marin-32B base model is less suited for Terminal-Corpus SFT than Qwen3-32B.

2. XLA compilation overhead: The Marin-32B model requires ~9 min XLA compilation per new request shape on TPU vLLM, eating most of the 900s agent timeout. The first task per shard always times out with 0 tokens. This reduces effective agent time to ~6 min per task.

3. Qwen3-32B's advantage may be architectural: The standard Qwen3 architecture (64 heads, no rope scaling) compiles faster on TPU, giving the agent more time. The Marin-32B's non-standard config (40 heads, llama3 rope scaling) causes longer compilation.

4. Coverage challenges: Preemptible TPUs and Daytona sandbox limits made full 89-task coverage difficult across all trials.

Artifacts

• Trial 1: gs://marin-us-east5/evaluation/harbor/terminal-bench/exp4760-marin32b-sft-step858/terminus-2/trial1/full/
• Trial 2: gs://marin-us-east5/evaluation/harbor/terminal-bench/exp4760-marin32b-sft-step858/terminus-2/trial2/full/
• Trial 3: gs://marin-us-east5/evaluation/harbor/terminal-bench/exp4760-marin32b-sft-step858/terminus-2/trial3/full/
• Model: gs://marin-us-east5/checkpoints/exp4760_sft_marin_32b_base_terminal_corpus_15pct_32768tok_v5p64-775003/hf/step-858
• Config: n_concurrent=1, tensor_parallel_size=4, max_model_len=32768, v5p-8

[AlienKevin]

🤖 TB2 Eval Results (corrected): Marin-32B SFT — all trials 89/89

Previous comment had incomplete coverage. All 3 trials reached full 89/89 tasks.

Updated Results

Trial	Tasks	Solved	Timeout	Errors	Failed (reward=0)
1	89/89	1 (1.1%)	72	16	0
2	89/89	0 (0.0%)	76	13	0
3	89/89	0 (0.0%)	85	4	0
Best across 3	89	1 (1.1%)

Sole solve: largest-eigenval (trial 1)

Comparison

Model	Solve rate (best trial)	Combined unique solves
Marin-32B SFT (exp4760, 15% data, 859 steps)	1/89 = 1.1%	1
Qwen3-32B SFT (exp4307, 100% data, 500 steps)	15/87 = 17.2%	20 (across 2 trials)

Failure analysis

• 0 tasks get reward=0 — the model never finishes a task wrong, it just times out
• ~9 min XLA compilation per task on TPU eats most of the 900s agent timeout
• The agent gets only ~6 min of effective work per task after compilation

[AlienKevin]

🤖 Root cause found: Degenerate repetitive generation

After extensive debugging of the Marin-32B SFT's TB2 eval failures, we identified the actual root cause by logging the raw vLLM response.

What the model actually generates

The model's first response to a TB2 task prompt (126,767 chars, hitting the 24K token output limit):

First ~2000 chars — coherent reasoning:

We are in a fresh terminal. The task is to recover a secret that was accidentally committed and then removed by rewriting history. The repo is at /app/repo. We need to recover the secret and write to /app/secret.txt, then clean up the secret in the repo... Maybe we can use git log --graph --all --decorate to see all commits, or git reflog, or git fsck to find dangling objects...

Remaining ~124,000 chars — degenerate repetition:

zorazorazorazorazorazazorazorazorazorazorazorazorzarazorazorazorzarazorazorzerazorazarazorazorazorazorazorazorazorazorazarazorazorazazorazorazorazorazazorazorazorazorazorazorazorazorazorazerazorazorazazora...

Previous hypotheses (ruled out)

Hypothesis	Status
vLLM TPU architecture incompatibility	❌ vLLM serves the model fine
XLA compilation timeout	❌ Model generates after compilation
Verbose thinking blocks from chat template	❌ No <|start_think|> tags in output
max_output_tokens too low (8192)	❌ Still degenerates with 24576 budget
max_model_len / max_input_tokens mismatch	❌ Secondary issue, not root cause

Actual root cause

The Marin-32B SFT model produces degenerate repetitive text after an initial coherent reasoning block. The model starts with sensible analysis (~2K chars) then collapses into an infinite repetition loop (zorazora...) that fills the entire output budget. It never produces the required JSON action format.

This is a fine-tuning quality issue, not an inference infrastructure issue. The Qwen3-32B SFT trained on the same Terminal-Corpus data does not exhibit this behavior — it produces 3-8K token responses with proper JSON formatting.

Possible causes:

• The marin-community/marin-32b-base model may be prone to repetitive degeneration after SFT
• Training on only 15% of the Terminal-Corpus (vs 100% for Qwen3) may not be sufficient for stable generation
• The Marin tokenizer + chat template combination may interact poorly with this base model

Debug artifacts

• Trial with logging: gs://marin-us-east5/evaluation/harbor/terminal-bench/exp4760-marin32b-sft-step858/terminus-2/debug-trunclog/
• The logging patch was added to lib/harbor/src/harbor/agents/terminus_2/terminus_2.py to capture truncated responses

[AlienKevin]

🤖 Inference settings experiments: repetition degeneration can be mitigated

Ran 5 parallel experiments on the git-leak-recovery task with different sampling parameters to test if the repetitive degeneration (zorazora...) can be broken.

Results

Exp	Temperature	top_k	rep_penalty	freq_penalty	Response	Degenerates?	Notes
A	0.9	50	-	-	133K chars	NO	Ends with "task_complete": true. 2nd call only 1.6K chars, coherent.
B	1.0	50	-	-	114K chars	NO	Ends with ", — likely JSON but very long
C	0.6	20	1.2	-	126K chars	YES	Still zorazora... — rep_penalty alone doesn't help at low temp
D	0.6	20	-	0.5	127K chars	YES	Still zorazora... — freq_penalty alone doesn't help at low temp
E	0.8	40	1.1	-	130K chars	NO	Ends coherently: "see what files are present."

Key findings

1. Higher temperature (≥0.8) prevents degeneration. Experiments A, B, and E (temp 0.8-1.0) all produce coherent text throughout. The default temp=0.6 is too low — the model enters low-entropy repetitive states.

2. Repetition/frequency penalties alone don't fix it. Experiments C and D (penalties with temp=0.6) still degenerate into zorazora.... The repetition loop uses enough token variety (5 unique tokens: azor, or, az, z, zar) that penalties can't break it at low temperature.

3. The model CAN produce valid output. Experiment A's first response contained "task_complete": true within the (truncated) output, and the second call was only 1,658 chars with coherent reasoning. With temp=0.9, the model works.

4. All responses are still very long (114K-133K chars, hitting 24K token limit). Even without degeneration, the model generates extremely long thinking blocks. This is consistent with the training data which has many 10K-30K char responses.

Recommendation

For future Marin-32B SFT evals on TB2:

• Use temperature ≥ 0.8 (0.9 recommended) to prevent degeneration
• Keep top_k ≥ 40 for sufficient diversity
• max_output_tokens of 8192 is too low — the model needs at least 16K, ideally 24K
• Repetition penalties are not effective substitutes for higher temperature

The model's verbosity (filling entire output budget) is a separate issue from degeneration — it's learned from the training data which has many 10K-30K char responses. The degeneration specifically is a sampling issue at low temperature with this model architecture.

[AlienKevin]

🤖 Final reevaluation result: Marin-32B SFT is broken.

Reran TB2 against exp4760-marin32b-sft-step858 with the proposed degeneration fix (temperature=0.9, top_k=50, top_p=0.95, max_output_tokens=24576, max_input_tokens=24576) across 10 shards.

Result

• Solves: 0 / 76 unique tasks attempted (122 trials, 85.4% TB2 coverage; 4/10 shards SUCCESS, 2 FAILED, 4 stuck RUNNING with no new results in ~40 min)
• AgentTimeoutError: 105 / 122 (86%), of which 86 (81.9%) carry OutputLengthExceededError in the traceback — i.e. the model degenerates and burns the entire 24576-token output budget without returning a tool call
• InternalServerError: 17 / 122 (vLLM-side failures)

GCS: gs://marin-us-east5/evaluation/harbor/terminal-bench/exp4760-marin32b-sft-step858/terminus-2/fixed-temp09/full/

Verdict

Raising temperature from 0.6 → 0.9 with top_k=50 did not break the degeneration. Combined with the prior runs (1/89 false-positive at original settings, 0/89 true), this rules out sampling as the cause. The fine-tuning itself is the failure mode — Marin-32B SFT produces degenerate output regardless of decoding parameters.

For comparison on the same TB2 setup (terminus-2 + Daytona):

Model	Solves	Pass rate
Qwen3-32B SFT (same dataset, same recipe)	15 / 87	17.2%
Marin-32B SFT (exp4760)	0 / 76	0.0%

Same Terminal-Corpus dataset, same SFT pipeline, same eval — only the base model differs. Reproduction failed on Marin-32B; the SFT recipe that works for Qwen3 does not work for Marin-32B as configured.

Suggested next steps

1. Investigate why the same SFT data produces healthy output on Qwen3-32B but degenerate output on Marin-32B (chat template + tokenizer were verified identical between training and inference, ruling that out).
2. Likely candidates to dig into: learning rate / loss-mask alignment under MARIN_CHAT_TEMPLATE, EOS-token handling for the Marin tokenizer, or under-trained checkpoint despite 858 steps × 2 epochs.

[AlienKevin]

🤖 Root cause identified — and fixed.

TL;DR

The exp4760 SFT was healthy. The saved checkpoint was misconfigured for inference: its chat template trained the model to emit <|eot_id|> (token 128009), but the saved config.json carried eos_token_id=128001 (<|end_of_text|>) inherited from marin-32b-base, and no generation_config.json was written. vLLM only stopped on 128001 — which the model never emits — so it kept sampling from a distribution it was never trained on, producing degenerate zorazora… repetition until the entire output budget was consumed (OutputLengthExceededError). The same condition explains all 86/105 timeouts in the temp=0.9 reeval.

This is purely a save/serve misconfiguration, not a training failure.

Why Qwen3-32B (#4307) doesn't hit this

• Qwen3 chat template emits <|im_end|>.
• Qwen3-32B base config already sets eos_token_id to that token.
• They match by construction — vLLM stops on the trained terminator.

For Llama-3-tokenizer-derived models like Marin, this alignment must be done explicitly. The canonical Llama-3-Instruct release does:

• config.json → eos_token_id: 128009
• generation_config.json → eos_token_id: [128001, 128009] (both work)
• tokenizer_config.json → eos_token: <|eot_id|>

Fixes landed

1. Patched the existing checkpoints in GCS (step-500, step-858):

• Wrote generation_config.json with eos_token_id: [128001, 128009]
• Updated config.json eos_token_id: 128001 → 128009
• Updated tokenizer_config.json eos_token: <|end_of_text|> → <|eot_id|>
• Existing safetensors are untouched — the model itself is fine; only metadata changed.

2. Plumbed generation_config through Levanter + Marin SFT pipeline so future runs save it automatically:

• HFCheckpointConverter gets a generation_config: dict | None field; if set, save_pretrained writes generation_config.json next to config.json.
• TrainLmConfig.generation_config → SimpleTrainConfig.generation_config → SimpleSFTConfig.generation_config plumbed end-to-end.
• Added MARIN_GENERATION_CONFIG = {"eos_token_id": [128001, 128009], "bos_token_id": 128000} to experiments/marin_models.py.
• experiments/exp4760_sft_marin_32b_base_terminal_corpus.py now passes generation_config=MARIN_GENERATION_CONFIG.

3. Defensive eval validator. experiments/exp4307_eval_released_nemotron_terminal_32b_tb2.py now calls _validate_chat_template_eos(model_path) at startup. For any gs://-style checkpoint, it reads chat_template.jinja + config.json + generation_config.json and raises RuntimeError if the template emits a Llama-3 turn terminator (<|eot_id|> / <|end_of_text|>) that isn't in the served stop tokens. HF Hub paths are skipped (we trust the upstream release). Verified end-to-end: passes on the patched checkpoint, raises on the pre-patch state and on a synthetic broken case.

Why fewer training steps than Qwen3 — answered

Not a bug — by design per this issue's description: "Post-train marin-community/marin-32b-base on 15% of Nemotron-Terminal-Corpus … against the Qwen3-32B SFT checkpoint at the same training fraction".

• exp4307 actually trained on 100% of the corpus (366,103 examples × 2 epochs / batch 128 ≈ 5,720 steps).
• exp4760 was authored as the 15% experiment (54,915 × 2 / 128 ≈ 858 steps), 6.7× shorter.

So the apples-to-apples Qwen3 baseline at 15% does not exist — the 17.2% Qwen3 result we've been comparing against is the 100% run. This is a comparison-design problem, separate from the EOS bug.

Suggested next steps:

1. Re-run the full TB2 eval against the now-patched step-858 checkpoint to get the real Marin-32B-at-15% number.
2. Decide whether to extend exp4760 to 100% data for an apples-to-apples comparison with exp4307, or train a 15% Qwen3-32B counterpart.

I'll kick off the patched-checkpoint reeval now and report back.