docs: retire Ray launcher references; route to Iris

[yonromai]

Summary

Stage 4 of the Ray-removal plan (#4453): sweep ray_run.py / scripts/ray/* / ray up / ray down references out of user-facing docs, agent skills, runbooks, and experiment docstrings, ahead of the production Ray cluster retirement. Replacements route readers to the live Iris launcher pattern documented in experiments/ferries/OPS.md:

uv run iris --cluster=marin job run ...

Out of scope (other retirement stages): scripts/ray/* code, infra/marin-*.yaml, infra/README.md "Maintaining a Ray Cluster" section, and lib/marin/src/marin/run/ray_run.py itself. Historical design/logbook docs (.agents/projects/20251114_fray_design.md, linear_ce_loss.md) are intentionally left alone — they document past state.

What was rewritten

• Docs (mkdocs): explanations/executor.md, explanations/{evaluation,experiments,guidelines,marin-prefix}.md, harbor-integration.md, references/resource-config.md, tutorials/{train-an-lm,train-dpo,storage-bucket,local-gpu,first-experiment,executor-101,installation}.md, recipes/add_scaling_heuristic.md.
• Agent content: .agents/skills/ferries/SKILL.md (daily launch command + "Ray job id" labels), .agents/skills/architecture/SKILL.md (entry point + infra prose), .agents/projects/ferry_framework.md (launch shape + run-record fields).
• Runbooks: experiments/tootsie/BABYSITTING.md — five launch snippets rewritten mechanically, preserving --force_run_failed / --run_only flags; experiments/grug/README.md; experiments/README_sft.md.
• Module docstrings: experiments/ferries/daily.py; experiments/tutorials/exp1077_reproduce_dclm_1b1x.py, exp1078_reproduce_dclm_7b1x.py; experiments/rollout_data/*.py (7 files with identical Usage: pattern).

What was deleted

• docs/dev-guide/rebuilding-cluster.md — entirely about ray up + scripts/ray/cluster.py update-configs rebuild flow.
• docs/tutorials/tpu-cluster-setup.md — whole tutorial is ray up / ray submit / ray dashboard; removed from mkdocs.yml nav; installation.md cross-link retargeted at lib/iris/OPS.md.
• lib/levanter/docs/design/Ray-Job-Manager.md — design doc for the Ray TPU job manager (implementation deleted in Delete Levanter Ray TPU infra #5031).
• .agents/docs/fray-migration.md — past migration plan, superseded.

Levanter Getting-Started-TPU-VM.md: removed the "Using the Ray Autoscaler" section (the code it described, launch_on_ray.py, was deleted in #5031). launch.py guidance kept; prose now points at lib/iris/OPS.md for Marin's shared-cluster path.

Iris doc verification

Before rewriting, I audited experiments/ferries/OPS.md and lib/iris/OPS.md end-to-end against the live iris CLI. Every subcommand and flag cited in those docs resolved. Specifically verified:

$ uv run iris --help            # top-level commands
$ uv run iris job run --help    # -e/--env-vars, --no-wait, --memory, --extra, etc.
$ uv run iris cluster --help    # start, stop, restart, dashboard, dashboard-proxy, status, vm, controller, list, start-smoke
$ uv run iris cluster controller --help   # checkpoint, restart, serve, worker-restart
$ uv run iris task exec --help  # TASK_ID COMMAND..., --timeout
$ uv run iris process --help    # status, logs, profile
$ uv run iris process profile --help  # threads|cpu|mem, -t target, -d duration
$ uv run iris rpc controller --help   # get-scheduler-state, get-autoscaler-status, etc.
$ uv run iris query --help      # -f table|json|csv
$ uv run iris cluster vm status --help   # --scale-group
$ uv run iris user budget --help         # get, list, set
$ uv run iris cluster list      # confirmed `marin` resolves to lib/iris/examples/marin.yaml

Every flag, example, and cross-reference in the two Iris docs matched the live CLI output. Referenced scripts exist (scripts/datakit/validate_ferry_outputs.py, .github/workflows/marin-datakit-smoke.yaml) and lib/iris/docs/priority-bands.md exists. No Iris-doc corrections required.

Test plan

• ./infra/pre-commit.py --all-files --fix passes (ruff, black, license headers, pyrefly, markdown, yaml, etc.).
• rg 'ray_run|scripts/ray/|ray up|ray down|ray submit|ray job submit|RAY_AUTH_TOKEN' docs/ lib/levanter/docs/ .agents/ experiments/ returns zero hits outside the two historical logbook files (.agents/projects/20251114_fray_design.md, linear_ce_loss.md) — verified via Grep tool.
• rg 'tpu-cluster-setup' returns zero hits (mkdocs nav + installation.md cross-link updated).
• rg 'Ray-Job-Manager|fray-migration' returns zero hits (both deleted files unreferenced).
• rg 'launch_on_ray' returns zero hits in lib/levanter/docs/.

Review handoff — tootsie runbook

@dlwh @Helw150 @rjpower — the experiments/tootsie/BABYSITTING.md rewrite in this PR is mechanical (preserves --force_run_failed / --run_only flags verbatim, switches launcher to uv run iris --cluster=marin job run -- python -m experiments.exp{600,750}_tootsie...). Please sanity-check the exact flags and cluster targeting — operators have right-of-refusal on the syntax. The doc also drops the manual_ray_worker_launch.py "reattach v4-2048" escape hatch and replaces it with the note that Iris handles preemption/restart itself. Speak up if that's not yet a safe claim for the big tootsie v4-2048 runs.

Refs: #4453 (parent Ray removal), #5029 (doc sweep tracker).

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 416fec329e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[yonromai]

🤖 Pushed 7fdce509 to address the P1 validator comment. --memory=4G → --memory=2G across all 18 stage-4 rewrites (matches the canonical experiments/ferries/OPS.md:17 pattern), plus lib/iris/OPS.md:42 corrected from --memory 16g → --memory 2g with a note about the 4 GB opt-in threshold.

[Helw150]

We can probably just delete this since all Tootsie's are done