Skip to content

Add ability to run "Free-Style" jobs in Optimus #407

Open
@sbchaos

Description

@sbchaos

Is your feature request related to a problem? Please describe.
Currently Optimus only provides a way to use a specific image in a job through the plugin, we require a way for people to run a free-style job where they can bring in their own image to be scheduled by the Optimus. This might address use-cases where creating a specific plugin might be an overkill or might not be feasible for Optimus team.

A typical use case might be to fetch logs after a job is run or run some other ad-hoc task, the new job can define a dependency on the job and fetch the logs after the previous job is complete.

Describe the solution you'd like
Providing a plugin to create a free-style job by specifying an image for the job to execute. This can have configuration like

version: 1
name: cleanup
owner: [email protected]
schedule:
  start_date: "2022-02-26"
behavior:
  depends_on_past: false
  catch_up: false
task:
  name: generic
  config:
    IMAGE: docker.io/my-log-extractor:2.3
    JOB_LABELS: owner=optimus
    PROJECT: integration
    LOCATION: /var/logs/my-system/job.log
    USER_NAME: "{{.secret.user}}"
    USER_TOKEN: "{{.secret.token}}"
    FORMAT: JSON
dependencies:
  - load_to_bigquery

This will run the image as non-root user, with the required config available as environment variables and assets made available.

Describe alternatives you've considered
Alternative might be to have a hook to external system to run these actions on completion of task. This might require people to manage their post jobs somewhere else.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions