Migrate all supported Marin code off Ray

[yonromai]

Migrate all supported Marin code off Ray

We're deprecating Ray in favor of Iris/Fray v2 (#4269). Execution, training, and Zephyr are already on v2, but a long tail of code still depends on Ray — either directly (import ray) or transitively via Fray v1 (which is Ray-backed). This issue tracks migrating every supported codepath so we can turn off the Ray cluster.

Ray entrypoints — code that needs migrating

These are library modules and workflows that consumers actively depend on. Each must be migrated to Fray v2 / Iris before Ray can be removed.

Direct import ray — needs architectural work (6 files)

Area	Files	What they use Ray for
Classification pipeline (3 files)	processing/classification/inference.py, classifier.py, autoscaler.py	@ray.remote actors, ray.util.queue.Queue, RayActorError — full actor-pool pattern for distributed inference
Executor (1 file)	execution/executor.py	ray.remote_function.RemoteFunction isinstance check + ray.get() — legacy support for @ray.remote-decorated executor steps
Levanter distributed (1 file)	levanter/distributed.py	ray.init(), ray.shutdown() — Levanter's distributed runtime init
vLLM server (1 file)	inference/vllm_server.py	from ray._private.accelerators import TPUAcceleratorManager in try/except — optional TPU detection

Via Fray v1 — needs import migration + current_cluster() → current_client() rework (24 files)

Area	Files	v1 APIs used
RL (7 files)	rl/rl_job.py, rl/train_worker.py, rl/rollout_worker.py, rl/curriculum.py, rl/weight_transfer/{arrow_flight,jax}.py, rl/scripts/evaluate_environment.py	current_cluster(), JobRequest, get_default_job_ctx()
Evaluation (9 files)	evaluation/evaluators/{evaluator,lm_evaluation_harness_evaluator,harbor_evaluator,simple_evaluator,levanter_tpu_evaluator,evalchemy_evaluator,levanter_lm_eval_evaluator}.py, evaluation/{visualize,run}.py	current_cluster(), JobRequest, build_runtime_env_for_packages
Processing	processing/classification/fasttext/train_fasttext.py	current_cluster(), JobRequest
Export	export/levanter_checkpoint.py	current_cluster(), JobRequest
Inference	inference/vllm_smoke_test.py	current_cluster(), JobRequest
Levanter	levanter/callbacks/_metrics.py	fray.v1.cluster.device_flops (trivial — v2 equivalent exists at fray.v2.device_flops)
Tests	tests/conftest.py, tests/transform/conftest.py, tests/datakit/download/conftest.py, tests/rl/test_weight_transfer.py, tests/rl/integration/test_iris_integration.py, tests/integration_test.py, lib/zephyr/tests/conftest.py	create_cluster, set_current_cluster, fray_default_job_ctx, ray.init(), @ray.remote in test

Note: evaluation/run.py imports both fray.v1.cluster.ResourceConfig and fray.cluster.ResourceConfig side by side with aliases — a hybrid state from incremental migration.

Ray orphans — delete, don't migrate (7 files)

These modules have no importers in library code. They're standalone CLI tools or dead code that will be deleted as part of #4269. No migration work needed.

File	Why it's orphan
run/ray_run.py	CLI entry point only. zephyr/cli.py shells out to it for zephyr ray submit, but that's a deprecated codepath.
scripts/ray/cluster.py	Standalone CLI for Ray cluster ops. No importers.
scripts/ray/dev_tpu.py	Standalone CLI for dev TPU allocation. No importers. Replaced by scripts/iris/dev_tpu.py.
levanter/infra/ray_tpu.py	Only imported by launch_on_ray.py (below).
levanter/infra/launch_on_ray.py	Only imports ray_tpu.py. No external callers. Dead chain.
marin/cluster/ray.py	Only imported by ray_run.py.
scripts/debug/inspect_data.py	Debug script with ray.init(), @ray.remote, JobSubmissionClient. No importers.

v1 APIs with no obvious v2 equivalent

These need design work, not just mechanical import swaps:

• get_default_job_ctx() — used pervasively in RL workers for peer discovery and actor creation. v2 uses current_client() but the job-context pattern is structurally different.
• build_runtime_env_for_packages() — Ray-specific runtime env builder used in evaluators. Concept doesn't apply to Iris.
• Direct @ray.remote actor pool — classification pipeline uses Ray's actor model directly. Needs redesign around Fray v2 ActorGroup.

Sequencing

1. Delete Ray orphans — the 7 files above with no library consumers
2. Migrate infra modules — Levanter callbacks (trivial v1→v2 swap), then Executor and Levanter distributed (need design work). These have broad downstream consumers (trainer, RL, evaluation, 100+ experiment files) so they must move first.
3. Migrate leaf modules — RL, Evaluation, Classification, Export, Inference, Fasttext. Can proceed in parallel once infra is ready.
4. Bake period — run all workflows off Ray
5. Remove fray.v1 and ray dependency from the repo

Related issues

• Single way of running jobs — off Ray completely #4269 — Off Ray completely (umbrella)
• Migrate RL pipeline from Fray v1 (Ray) to Fray v2 (Iris) #3959 / [RL] Migrate single-host RL pipeline to Fray v2/Iris #3960 — RL pipeline migration to Fray v2/Iris
• iris: migrate classification/inference to Fray & Iris #4088 / [classification] Migrate inference pipeline from Ray to Fray v2 #4090 — Classification inference migration
• Migrate logprob evals to Fray v2 + some flybys #4398 — Logprob evals migration (merged)

Definition of done

• Zero import ray in lib/marin/src/marin/ (excluding test helpers)
• Zero from fray.v1 imports in lib/marin/src/marin/
• Zero from fray.v1 imports in lib/levanter/src/levanter/
• Test fixtures in tests/ use v2 APIs
• Ray orphan files deleted (tracked in Single way of running jobs — off Ray completely #4269)
• .agents/docs/fray-migration.md updated or deleted (currently stale)
• No regressions in eval, RL, classification, or export workflows

[yonromai]

🤖 Pyrefly-backed remaining Ray-dependent workloads after support-layer removal

After removing Ray from uv and deleting the Ray-specific support layers on updated main, strict pyrefly still reports 21 Ray-related errors across 12 workload files. These are the remaining supported workloads that still need a port-or-remove decision.

Area	Path(s)	Remaining workload surface	Remaining Ray dependency	Likely owner(s)
Levanter cache builder	lib/levanter/src/levanter/main/cache_dataset.py	1 file	RayConfig	David Hall
Evaluation: LM eval launchers	lib/marin/src/marin/evaluation/evaluators/levanter_lm_eval_evaluator.py lib/marin/src/marin/evaluation/evaluators/lm_evaluation_harness_evaluator.py	2 files	RayConfig, deleted fray.v1.cluster.ray.deps, ray=...	Tony Lee, Nikil Ravi
Evaluation: logprob jobs	lib/marin/src/marin/evaluation/log_probs.py lib/marin/src/marin/evaluation/save_logprobs.py lib/marin/src/marin/evaluation/visualize.py	3 files	RayConfig, ray=...	Rohith Kuditipudi, David Hall
Classification inference	lib/marin/src/marin/processing/classification/autoscaler.py lib/marin/src/marin/processing/classification/classifier.py lib/marin/src/marin/processing/classification/inference.py	3 files	direct ray, ray.exceptions, ray.util.queue	Christopher Chou
RL helpers	lib/marin/src/marin/rl/rl_experiment_utils.py lib/marin/src/marin/rl/scripts/evaluate_environment.py	2 files	RayConfig, ray=...	Kevin Xiang Li, Ahmed Ahmed, Russell Power
Inference helper	lib/marin/src/marin/inference/vllm_server.py	1 file	ray._private.accelerators	David Hall

Method and scope

• Ran on updated main in a fresh worktree.
• Removed Ray from uv metadata and lockfile.
• Deleted Ray-specific support code: Fray Ray backends, Marin Ray cluster/run glue, Levanter Ray infra, scripts/ray, and Ray cluster configs.
• Reran strict pyrefly after those deletions.
• Scope here is the pyrefly-checked library surface. This is not a repo-wide grep, and it does not claim to cover experiments, scripts, or tests outside that surface.

Exact surviving workload files and pyrefly evidence

Levanter cache builder

• lib/levanter/src/levanter/main/cache_dataset.py
  • likely owner: David Hall
  • missing-module-attribute: RayConfig

Evaluation: LM eval launchers

• lib/marin/src/marin/evaluation/evaluators/levanter_lm_eval_evaluator.py
  • likely owner: Nikil Ravi
  • missing-module-attribute: RayConfig
  • missing-import: fray.v1.cluster.ray.deps
  • unexpected-keyword: ray=...
• lib/marin/src/marin/evaluation/evaluators/lm_evaluation_harness_evaluator.py
  • likely owner: Tony Lee
  • missing-import: fray.v1.cluster.ray.deps

Evaluation: logprob jobs

• lib/marin/src/marin/evaluation/log_probs.py
  • likely owners: Rohith Kuditipudi, David Hall
  • missing-module-attribute: RayConfig
  • unexpected-keyword: ray=...
• lib/marin/src/marin/evaluation/save_logprobs.py
  • likely owner: Rohith Kuditipudi
  • missing-module-attribute: RayConfig
  • unexpected-keyword: ray=...
• lib/marin/src/marin/evaluation/visualize.py
  • likely owners: David Hall, Rohith Kuditipudi
  • missing-module-attribute: RayConfig
  • unexpected-keyword: ray=...

Classification inference

• lib/marin/src/marin/processing/classification/autoscaler.py
  • likely owner: Christopher Chou
  • missing-import: ray
  • missing-import: ray.exceptions
  • missing-import: ray.util.queue
• lib/marin/src/marin/processing/classification/classifier.py
  • likely owner: Christopher Chou
  • missing-import: ray
• lib/marin/src/marin/processing/classification/inference.py
  • likely owner: Christopher Chou
  • missing-import: ray
  • missing-import: ray.util.queue

RL helpers

• lib/marin/src/marin/rl/rl_experiment_utils.py
  • likely owners: Kevin Xiang Li, Ahmed Ahmed
  • missing-module-attribute: RayConfig
  • unexpected-keyword: ray=...
• lib/marin/src/marin/rl/scripts/evaluate_environment.py
  • likely owners: Russell Power, Ahmed Ahmed
  • missing-attribute: levanter.distributed.RayConfig

Inference helper

• lib/marin/src/marin/inference/vllm_server.py
  • likely owners: David Hall, Russell Power
  • missing-import: ray._private.accelerators

How I inferred likely owners

• Looked only at the surviving workload files above, not the broader modules from the earlier archaeology comment.
• Used git log --follow and author concentration on those files.
• Weighted recent domain-specific edits more heavily than cross-cutting refactors, bot commits, and license-header churn.
• These are intended as likely contacts for the port-or-remove decision, not hard ownership claims.

[yonromai]

For context: Had a chat with Helw150, next steps for ^:

• Reach out to:
  • dlwh about "Levanter cache builder" and "Inference helper": Do Levanter cache dataset and vllm server need to be ported to Iris? #4641
  • rohit about "Evaluation: logprob jobs": Does "Evaluation: logprob jobs" need to be ported to Iris? #4640
  • Ahmed & Kevin about "RL helpers": Do "RL helpers" Need to be ported to Iris? #4639
• Scope out work for: "Evaluation: LM eval launchers"
• Delete "Classification inference", see if anyone complains: [byebye-ray] Delete inference and fasttext tooling #4642