[Flyte-7005] Supporting Ray autoscalerOptions by 0yukali0 · Pull Request #7111 · flyteorg/flyte

0yukali0 · 2026-03-30T03:09:30Z

Tracking issue

Related #7005

Why are the changes needed?

Ray autoScalerOptions is not supportted in flyte v1

What changes were proposed in this pull request?

Add autoscalerOptions to ray.proto
Generate go files of grpc proto
Parsing the options from grpc and translate it to crd of ray operator.

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

added: For new features.
changed: For changes in existing functionality.
deprecated: For soon-to-be-removed features.
removed: For features being removed.
fixed: For any bug fixed.
security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

clone flyte and flytekit (branch flyte-7005 and flytekit-7005)
run the flyte backend in sandbox
2.1 flytectl demo start --dev
2.2 make compile && POD_NAMESPACE=flyte ./flyte start --config flyte-single-binary-local.yaml
Update ray example like Ray example
3.1 uv venv -p 3.11 && source .venv/bin/activate && uv pip install -e ~/flytekit ~/flytekit/plugins/flytekit-ray && uv pip install -e ~/flyte/flyteidl
3.2 "pyflyte run --remote ray_example.py ray_workflow --n 10"

Screenshots

I updated the documentation accordingly.
All new and existing tests passed.
All commits are signed-off.

Related PRs

Docs link

codecov · 2026-03-30T03:13:10Z

Codecov Report

❌ Patch coverage is 82.60870% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.96%. Comparing base (682d81e) to head (4265b65).

Files with missing lines	Patch %	Lines
flyteplugins/go/tasks/plugins/k8s/ray/ray.go	82.60%	4 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #7111   +/-   ##
=======================================
  Coverage   56.95%   56.96%           
=======================================
  Files         931      931           
  Lines       58250    58273   +23     
=======================================
+ Hits        33179    33197   +18     
- Misses      22017    22022    +5     
  Partials     3054     3054

Flag	Coverage Δ
unittests-datacatalog	`53.51% <ø> (ø)`
unittests-flyteadmin	`53.14% <ø> (ø)`
unittests-flytecopilot	`43.06% <ø> (ø)`
unittests-flytectl	`64.09% <ø> (ø)`
unittests-flyteidl	`75.71% <ø> (ø)`
unittests-flyteplugins	`60.24% <82.60%> (+0.04%)`	⬆️
unittests-flytepropeller	`53.71% <ø> (ø)`
unittests-flytestdlib	`62.58% <ø> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

machichima · 2026-04-01T09:56:04Z

Hi @0yukali0,
Could you briefly describe how you test and provide the test script?
Thank you!

0yukali0 · 2026-04-02T13:47:50Z

Hi @machichima , i updated the setup steps and screenshot in the PR descriptions.
i am going to add the resources field in headspec and workgroupspec to replace requests and limits in next commit.

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

0yukali0 · 2026-04-11T01:43:57Z

with kuberay 1.1.0 and 1.5.1，run ray task with 2.54.1 is worked.

machichima · 2026-04-21T01:54:25Z

+			resourceRequirements := v1.ResourceRequirements{}
+			if requests := res.GetRequests(); len(requests) > 0 {
+				resourceRequirements.Requests = convertResourceEntriesToResourceList(requests)
+			}
+			if limits := res.GetLimits(); len(limits) > 0 {
+				resourceRequirements.Limits = convertResourceEntriesToResourceList(limits)
+			}


Could we use flytek8s.ToK8sResourceRequirements here instead?

flyte/flyteplugins/go/tasks/pluginmachinery/flytek8s/utils.go

Lines 52 to 68 in 6e84ac2

func ToK8sResourceRequirements(resources *core.Resources) (*v1.ResourceRequirements, error) {

res := &v1.ResourceRequirements{}

if resources == nil {

return res, nil

}

req, err := ToK8sResourceList(resources.GetRequests())

if err != nil {

return res, err

}

lim, err := ToK8sResourceList(resources.GetLimits())

if err != nil {

return res, err

}

res.Limits = lim

res.Requests = req

return res, nil

}

machichima · 2026-04-21T01:56:01Z

+			autoScalarOptions.Env = []v1.EnvVar{}
+			for _, env := range envs {
+				name := env.GetKey()
+				if val := env.GetValue(); val != "" {
+					autoScalarOptions.Env = append(autoScalarOptions.Env, v1.EnvVar{
+						Name:  name,
+						Value: val,
+					})
+				}
+			}


Could we use flytek8s.ToK8sEnvVar instead?

flyte/flyteplugins/go/tasks/pluginmachinery/flytek8s/utils.go

Line 12 in 6e84ac2

func ToK8sEnvVar(env []*core.KeyValuePair) []v1.EnvVar {

machichima · 2026-04-21T01:59:27Z

+		idleTimeoutTime := options.GetIdleTimeoutSeconds()
+		autoScalarOptions.IdleTimeoutSeconds = &idleTimeoutTime


If the idleTimeoutTime is not set (= 0), this part will overwrite the default value (60 sec) set in kuberay. we should only set autoScalarOptions.IdleTimeoutSeconds if it's not 0 (not empty)

https://github.com/ray-project/kuberay/blob/0706043ae00a0dedfadead423926136e01daaef9/proto/cluster.proto#L145-L147

https://github.com/ray-project/ray/blob/db711158e25231b77ef2fb968320b2319270732d/python/ray/autoscaler/_private/kuberay/autoscaling_config.py#L108-L111

Hi @machichima
I will update it and the following test case to check the setting.

machichima · 2026-04-21T02:01:18Z

 }

+func buildAutoscalerOptions(options *plugins.AutoscalerOptions) *rayv1.AutoscalerOptions {
+	var autoScalarOptions *rayv1.AutoscalerOptions


Suggested change

var autoScalarOptions *rayv1.AutoscalerOptions

var autoScalerOptions *rayv1.AutoscalerOptions

nit

machichima · 2026-04-21T02:03:13Z

+	}
+}
+
+func TestNewAutoscalerOptions(t *testing.T) {


Suggested change

func TestNewAutoscalerOptions(t *testing.T) {

func TestBuildAutoscalerOptions(t *testing.T) {

nit

machichima · 2026-04-21T02:05:35Z

+		assert.Equal(t, resource.MustParse("1Gi"), result.Resources.Limits[corev1.ResourceMemory])
+	})
+
+	t.Run("env literal value", func(t *testing.T) {


Could we add a test for env with empty value (should be skipped)?

flytek8s.ToK8sEnvVar allows empty value, so i will skip the test with empty value.

machichima · 2026-04-21T02:14:54Z

+		assert.Nil(t, buildAutoscalerOptions(nil))
+	})
+
+	t.Run("idle timeout propagated", func(t *testing.T) {


Could we also add a test like this:

t.Run("idle timeout zero should not be set", func(t *testing.T) { result := buildAutoscalerOptions(&plugins.AutoscalerOptions{}) require.NotNil(t, result) assert.Nil(t, result.IdleTimeoutSeconds) })

Which is related to my comment above: https://github.com/flyteorg/flyte/pull/7111/changes#r3114611720
If the IdleTimeoutSeconds is not set, it should remain empty so that Ray can apply its default value

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

pingsutw · 2026-04-23T21:55:13Z

+func convertResourceEntriesToResourceList(entries []*core.Resources_ResourceEntry) v1.ResourceList {
+	resourceList := v1.ResourceList{}
+	for _, entry := range entries {
+		var name v1.ResourceName
+		switch entry.GetName() {
+		case core.Resources_CPU:
+			name = v1.ResourceCPU


Do we use it anywhere? if not, can we remove it

pingsutw · 2026-04-23T22:00:17Z

+  // custom autoscaler image
+  string image = 4;
+  // autoscaler container resources
+  core.Resources resources = 5;


Should we use core.K8sPod k8s_pod instead of core.Resources? SDk can use this podSpec to override to resource, label or annotation. same as HeadGroupSpec and WorkerGroupSpec

No, AutoscalerOptions can't follow same solution as HeadGroupSpec and WorkerGroupSpec, the reason is following.
The api doc describes that AutoscalerOptions currently only support corev1.ResourceRequirements when HeadGroupSpec and WorkerGroupSpec support corev1.PodTemplateSpec.

Same situation also exists in v1 version

Signed-off-by: yuteng chen <a08h0283@gmail.com>

0yukali0 force-pushed the flyte-7005 branch 2 times, most recently from 951fbeb to 8a99445 Compare March 31, 2026 14:07

0yukali0 marked this pull request as ready for review March 31, 2026 14:45

0yukali0 force-pushed the flyte-7005 branch from 97559c4 to e5905e7 Compare April 1, 2026 07:32

machichima reviewed Apr 1, 2026

View reviewed changes

0yukali0 force-pushed the flyte-7005 branch from 57f9ebf to c9c99d1 Compare April 2, 2026 03:32

0yukali0 added 13 commits April 6, 2026 14:19

Add autoscaleroptions of proto files

3f3acef

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

fix EnvVar

9eaead8

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

fix resource

57cfe74

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

generate go from proto

841ae94

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

Add options to ray cluster request

7ed7786

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

update proto

4faca42

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

envvar added

15163b5

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

add resource

5679b2b

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

meeting lint

3267152

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

add unit test of new functions

92526f3

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

remvoe envfrom

ce0fceb

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

remote envfrom condition and test

e21d69c

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

regen and fix bugs

6e84ac2

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

0yukali0 force-pushed the flyte-7005 branch from c9c99d1 to 6e84ac2 Compare April 6, 2026 06:19

machichima reviewed Apr 21, 2026

View reviewed changes

0yukali0 added 2 commits April 21, 2026 21:36

set idletime if it is required

4bb66fb

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

adopting k8s utils function

f4429c9

Signed-off-by: Yuteng Chen <a08h0283@gmail.com>

pingsutw reviewed Apr 23, 2026

View reviewed changes

remove unused func

0061d44

Signed-off-by: yuteng chen <a08h0283@gmail.com>

Merge branch 'master' into flyte-7005

4265b65

pingsutw approved these changes Apr 24, 2026

View reviewed changes

pingsutw merged commit 2d13c8f into flyteorg:master May 9, 2026
49 checks passed

	func ToK8sResourceRequirements(resources core.Resources) (v1.ResourceRequirements, error) {
	res := &v1.ResourceRequirements{}
	if resources == nil {
	return res, nil
	}
	req, err := ToK8sResourceList(resources.GetRequests())
	if err != nil {
	return res, err
	}
	lim, err := ToK8sResourceList(resources.GetLimits())
	if err != nil {
	return res, err
	}
	res.Limits = lim
	res.Requests = req
	return res, nil
	}

		idleTimeoutTime := options.GetIdleTimeoutSeconds()
		autoScalarOptions.IdleTimeoutSeconds = &idleTimeoutTime

	var autoScalarOptions *rayv1.AutoscalerOptions
	var autoScalerOptions *rayv1.AutoscalerOptions

	func TestNewAutoscalerOptions(t *testing.T) {
	func TestBuildAutoscalerOptions(t *testing.T) {

Uh oh!

Conversation

0yukali0 commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Tracking issue

Related #7005

Why are the changes needed?

What changes were proposed in this pull request?

How was this patch tested?

Labels

Setup process

Screenshots

Related PRs

Docs link

Uh oh!

codecov Bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

machichima commented Apr 1, 2026

Uh oh!

0yukali0 commented Apr 2, 2026

Uh oh!

0yukali0 commented Apr 11, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

0yukali0 commented Mar 30, 2026 •

edited

Loading

codecov Bot commented Mar 30, 2026 •

edited

Loading