Skip to content

Provide a way to implement a MultiKueue support for the custom Jobs #2349

@tenzen-y

Description

This is blocked by

What would you like to be added:
I would like to support the extension mechanism so that we can implement the MultiKueue controller for the custom / in-house Jobs. I'm thinking of exposing the jobAdapter interface and implementing a mechanism to add arbitrary objects to the adapters similar to the jobframework integration manager:

adapters:

adapters = map[string]jobAdapter{
batchv1.SchemeGroupVersion.WithKind("Job").String(): &batchJobAdapter{},
jobset.SchemeGroupVersion.WithKind("JobSet").String(): &jobsetAdapter{},
}

jobAdapters interface:
type jobAdapter interface {
// Creates the Job object in the worker cluster using remote client, if not already created.
// Copy the status from the remote job if already exists.
SyncJob(ctx context.Context, localClient client.Client, remoteClient client.Client, key types.NamespacedName, workloadName, origin string) error
// Deletes the Job in the worker cluster.
DeleteRemoteObject(ctx context.Context, remoteClient client.Client, key types.NamespacedName) error
// KeepAdmissionCheckPending returns true if the state of the multikueue admission check should be
// kept Pending while the job runs in a worker. This might be needed to keep the managers job
// suspended and not start the execution locally.
KeepAdmissionCheckPending() bool
// IsJobManagedByKueue returns:
// - a bool indicating if the job object identified by key is managed by kueue and can be delegated.
// - a reason indicating why the job is not managed by Kueue
// - any API error encountered during the check
IsJobManagedByKueue(ctx context.Context, localClient client.Client, key types.NamespacedName) (bool, string, error)
}

jobframework integration manager: https://github.com/kubernetes-sigs/kueue/blob/e461fe0827786e5cd6f45ff6739ebeed9a700b05/pkg/controller/jobframework/integrationmanager.go

Why is this needed:
In many company, they have company specific CustomResources, and we should give a possibility to manage such CustomResources across multiple clusters.

Completion requirements:

This enhancement requires the following artifacts:

  • Design doc
  • API change
  • Docs update

The artifacts should be linked in subsequent comments.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

kind/featureCategorizes issue or PR as related to a new feature.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions