Delete Ray operator tooling

[yonromai]

Stage 5 of the Ray retirement plan (#4453). Deletes the Ray operator scripts, the Ray cluster Dockerfile, and the GitHub Actions / Makefile plumbing that built and tested them. Colleague sign-off received.

Summary

Files deleted

• scripts/ray/cluster.py (~1040 LOC) — Ray cluster lifecycle wrapper.
• scripts/ray/dev_tpu.py — superseded by scripts/iris/dev_tpu.py.
• scripts/ray/cleanup_workers.py — Ray-specific preempted-TPU cleanup.
• scripts/ray/README.md.
• scripts/debug/inspect_data.py — Ray-only data-inspection CLI.
• docker/marin/Dockerfile.cluster — based on rayproject/ray:2.53.0-py311-cpu, audit confirmed Ray-only.
• .github/workflows/marin-cleanup-tpus.yaml — invoked scripts/ray/cleanup_workers.py on cron.
• The marin-cluster-images job in .github/workflows/docker-images.yaml (build, smoke test, push, and the auto-PR that regenerated Ray cluster configs). Other jobs in the file (Iris, TPU CI, Levanter) are untouched.
• cluster_docker* / cluster_tag Makefile targets (and their .PHONY entries). These only fed Dockerfile.cluster.

Helpers moved (not duplicated)

scripts/debug/inspect_data.py defined two small helpers — _normalize_cluster_region and _validate_data_region — that were only imported by tests/test_inspect_data_region.py. Rather than find a new home for them, they move into the test file itself (their sole importer). No behavior change.

• Before: from scripts.debug.inspect_data import _normalize_cluster_region, _validate_data_region
• After: helpers defined at the top of tests/test_inspect_data_region.py.

Audit findings — lib/marin/src/marin/cluster/{config,gcp}.py NOT deleted

The plan's stage 5 table said these modules ""become dead once scripts/ray/cluster.py is gone"" and asked for a re-grep first. The re-grep turned up live callers outside Ray operator tooling, so per the plan's explicit ""stop and report"" directive I am leaving both files in place this stage.

$ rg 'marin\.cluster\.(config|gcp)|from marin\.cluster import' lib/ experiments/ scripts/ tests/
scripts/debug/inspect_data.py              — DELETED this PR
scripts/ray/cluster.py                     — DELETED this PR
scripts/ray/dev_tpu.py                     — DELETED this PR
scripts/gcp-ssh                            — LIVE, uses marin.cluster.gcp (get_project_id, find_tpu_by_ip, find_vm_by_ip, ssh_to_vm, ssh_to_tpu)
scripts/iris/dev_tpu.py                    — LIVE, uses marin.cluster.gcp (get_project_id, find_tpu_by_ip, find_vm_by_ip)
lib/marin/src/marin/run/ray_run.py         — LIVE, uses marin.cluster.config.find_config_by_region; queued for deletion in stage 3h

Concretely:

• lib/marin/src/marin/cluster/config.py — still imported by lib/marin/src/marin/run/ray_run.py, which is the stage 3h target. Once 3h lands this file has no callers and can go.
• lib/marin/src/marin/cluster/gcp.py — mixed module. Ray-specific functions (terminate_tpus_in_cluster, terminate_head_node, both label-filtered on ray-node-*) are dead after this PR, but the generic gcloud wrappers (get_project_id, find_tpu_by_ip, find_vm_by_ip, ssh_to_vm, ssh_to_tpu, list_instances, list_tpu_nodes, delete_tpu_node, run_gcloud_command) have active non-Ray consumers in scripts/gcp-ssh and scripts/iris/dev_tpu.py. A follow-up should split the Ray-specific helpers out and keep the rest.

I suggest a follow-up after 3h lands: delete config.py and split gcp.py into Ray-only (deleted) and generic (kept, possibly relocated out of the cluster/ package).

Test plan

• rg 'import ray' lib/ scripts/ — only lib/fray/** hits remain:

  lib/fray/src/fray/v1/cluster/ray/cluster.py
  lib/fray/src/fray/v1/cluster/ray/resources.py
  lib/fray/src/fray/v1/cluster/ray/tpu/execution.py
  lib/fray/src/fray/v1/job/context.py          (guarded)
  lib/fray/src/fray/v2/client.py               (guarded, line 192)
  lib/fray/src/fray/v2/ray_backend/backend.py
  lib/fray/src/fray/v2/ray_backend/context.py
  lib/fray/src/fray/v2/ray_backend/dashboard.py
  lib/fray/src/fray/v2/ray_backend/resources.py
  lib/fray/src/fray/v2/ray_backend/tpu.py
  

• uv sync — succeeds. The root package has no tpu extra; uv sync --package marin --extra tpu resolves 597 packages on macOS but cannot install libtpu wheels (no arm64 macOS wheels exist; unrelated to this change).
• ./infra/pre-commit.py --all-files --fix — passes end-to-end: Ruff, Black, license headers, Pyrefly, YAML/TOML, trailing whitespace, EOF newline, Jupyter, large files, Python AST, merge conflicts, markdown.
• Migrated helpers still tested: uv run --with pytest pytest tests/test_inspect_data_region.py -v — 7 passed in 0.27s.

Parent issue: #4453.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7362d660f6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Update Ray build docs when removing cluster_docker targets

Removing the cluster_docker*/cluster_tag Make targets breaks the documented operator flow immediately, because docs/dev-guide/rebuilding-cluster.md (lines 60-70) and infra/README.md (lines 224-247) still instruct users to run those targets and scripts/ray/cluster.py, which this commit also deletes. In practice this now yields make: *** No rule to make target ... and dead commands in runbooks, so the docs are no longer executable after this change; this also violates the root AGENTS.md rule to keep MkDocs docs in sync with code.

Useful? React with 👍 / 👎.