Skip to content

Add a new containers property to the workflow resources #998

Open
@cdavernas

Description

@cdavernas

What Would You Like to See Added?

Introduce a new containers property to the workflow's resources.

Why Is This Needed?

Currently, both shell and script processes are expected to run within the workflow's environment or platform, which can vary significantly from one runtime to another.

For example:

  • One runtime might execute a single container per workflow, utilizing embedded processors for each defined task.
  • Another runtime might opt to run each task in a different container.

This flexibility is crucial, as we should not impose constraints on any runtime, especially regarding their architectural choices. The diversity in implementations for shell and script tasks can lead to undesired side effects or even total incompatibility. Consider the following scenarios:

  • Shell scripts running in Windows versus Linux environments.
  • Python scripts executed in Python 2 versus Python 3.

One potential solution is to "jail" these tasks by requiring authors to specify a specific environment for each. While this approach is acceptable, it introduces new challenges, such as limiting the ability to run multiple commands or scripts successively within the same container—similar to the capabilities offered by GitHub Actions. Achieving functionality akin to GitHub Actions would necessitate significant and perhaps unnecessary refactoring of the domain-specific language (DSL).

Proposed Solution

I propose allowing authors to define top-level, reusable containers that can be assigned to script and shell tasks. This approach would:

  • "Jail" the environment of these tasks.
  • Enhance reusability, minimizing performance and cost impacts for most use cases.

Additionally, this proposal would address issue #874 by enabling authors to specify which versions of a script they wish to run, without being restricted to specific ones, as suggested in our weekly discussions.

Example Implementation

The following YAML illustrates the proposed structure:

document:
  dsl: '1.0.0-alpha1'
  namespace: default
  name: jail-shell-and-script-tasks
  version: '0.1.0'
use:
  containers:
    alpine:
      image: alpine:latest
do:
  - greetTheWorld:
      run:
        shell:
          command: '@echo "Hello, World!"'
        on: alpine  # Using a predefined container
  - greetTheWorldASecondTimeInSameContainer:
      run:
        shell:
          command: '@echo "Hello again, World!"'
        on: alpine  # Using the same predefined container as before
  - greetTheWorldUsingPython:
      run:
        script:
          code: >
            print('Hello, World!')
        on:
          image: python:3.10  # Using a container defined in-line

Conclusion

With this proposal, script and shell tasks would effectively become shorthand for container tasks and could potentially be transformed into cataloged custom functions. This change would lead to a more consistent and flexible approach for executing tasks within workflows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Stale Issuearea: ctkChanges in the CTK (Compliance Test Kit)area: specChanges in the SpecificationbacklogThe issue has been recognized but not yet scheduled or prioritizedchange: breakingA breaking change that will impact in a major version change.change: documentationImprovements or additions to documentation. It won't impact a version change.change: featureNew feature or request. Impacts in a minor version change

    Type

    No type

    Projects

    Status

    Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions