Migrate to RayJobs instead of managing RayClusters #1337
Description
What is the expected enhancement?
From the Kuberay docs: "With RayJob, KubeRay automatically creates a RayCluster and submits a job when the cluster is ready. You can also configure RayJob to automatically delete the RayCluster once the Ray job finishes."
We currently spin up a RayCluster when a user submits a job and have a decent amount of code dedicated to managing said cluster. Now that RayJob has reached v1, we should consider migrating to use RayJobs, which will offload the cluster management piece, hopefully simplifying our code base.
More on RayJobs: https://docs.ray.io/en/master/cluster/kubernetes/getting-started/rayjob-quick-start.html