Skip to content

[feat] Add DAG conditions and composable sub-workflows #1078

Closed
#1129
Closed
[feat] Add DAG conditions and composable sub-workflows#1078
#1129
Assignees
SaulMoro

Description

@hsuchifeng
hsuchifeng
opened on Oct 13, 2026

Parent: #818

Goal

Add deterministic acyclic control flow and workflow composition without exposing the graph to drivers.

Scope

  • Add needs, if, inputs, with, outputs, file and inline workflow, ends, and end.
  • Add env: mappings for command steps; mapped values become environment variables.
  • Implement a restricted expression parser; do not use JavaScript eval.
  • Support templates over inputs, enclosing values, and settled step results, but do not expand templates inside a command string.
  • Define success, failure, stopped, and skipped propagation.
  • Implement the proposed join rule: a step with several dependencies is skipped only when every dependency was skipped.
  • Permit recovery branches whose condition reads a failed dependency.
  • Isolate file sub-workflow inputs while allowing inline workflows to see their enclosing scope.
  • Validate missing references, cycles, recursion, unreachable steps, invalid outputs, and unreachable declared ends per file.
  • Keep all graph traversal and expansion in the engine.
  • Extend the validated workflow cache without changing [feat] Add a minimal command-only workflow runtime #1076’s per-file failure isolation.

Out of scope

Loops, break, for_each, approvals, workspaces, triggers, event waits, goto, switch, and needs_any.

Dependencies

Acceptance criteria

  • Linear, diamond, recovery, inline, and file sub-workflow cases settle deterministically.
  • Invalid references and graph cycles identify the file and step.
  • A named end settles the current workflow and is exposed to its parent.
  • Drivers require no graph-specific capability.
  • Expressions cannot access arbitrary globals, files, environment variables, or executable code.
  • Command inputs are available only through declared env: mappings; command strings are not template-expanded, and an input such as $(...) is not executed by the shell.

Verification

Focused graph, join, propagation, template, command-env, input/output, and ending tests; build/typecheck/lint; one command-only real-CLI E2E across a file sub-workflow. Update both workflow guide languages.

Activity

  1. hsuchifeng
    added a parent issue on Oct 13, 2026
  2. hsuchifeng
    marked this as blocked on Oct 13, 2026
  3. hsuchifeng
    marked this as blocking 5 issues on Oct 13, 2026

  4. SaulMoro commented on Oct 13, 2026

    @SaulMoro
    SaulMoro
    on Oct 13, 2026
    Collaborator

    How do inputs reach a command step? #818 had an env: key, so values go in as variables and never into the shell string. I don't see it here, and #1082 mentions command inputs shadowing secrets, so maybe you have another way in mind. If templates are meant to expand inside command too, that's the case #818 tried to avoid: an input like $(...) would run as shell.

  5. hsuchifeng commented on Oct 14, 2026

    @hsuchifeng
    hsuchifeng
    on Oct 14, 2026
    CollaboratorAuthor

    Inputs reach a command step only through an env: key. Those values become environment variables; the command string is not template-expanded, so an input such as $(...) is not executed by the shell.

    #1082’s secret-shadowing rule applies to those environment-variable names.

  6. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    SaulMoro
    on Oct 14, 2026
    Collaborator

    References by name (workflow: review-and-fix here, agent: and prompt_file from #1077) could resolve through the namespace rule the other resources already follow in src/namespace-resolver.ts. A namespace item replaces the root item of the same name, and one name in two active namespaces is a conflict. With that rule in place, #1101 adding workflows/<ns>/ does not change what an existing reference points to.

  7. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    SaulMoro
    on Oct 14, 2026
    Collaborator

    @hsuchifeng Can I assign this ticket to myself?

  8. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    SaulMoro
    on Oct 14, 2026
    Collaborator

    I'm taking this one. A few points the ticket leaves open or that I narrowed while planning it:

    • needs after the first step is required. needs: [] declares another root. Without this, test then merge with no needs would merge after a failed test. Every example in Discussion: Workflow Runtime Integration #818 already follows the rule, and relaxing it later breaks nothing.
    • Recovery. A stopped dependency skips a step unless its if reads that dependency, and the if has to read every stopped one. Other branches keep settling. The run takes the outcome of the end it reached; with no end, it fails if any step stopped.
    • File sub-workflows resolve by name from the validated cache. Pull checks across files for a missing child or recursion and reports it on the parent's step. start snapshots the whole tree into the run, and a child with no runnable version fails start by name, so nothing falls back to an older child. Scanning workflows/<ns>/ stays with [feat] Make all distributed resources pushable with project and role namespaces #1101. Once the sync applies the namespace rule, references follow it with no engine change.
    • Top-level inputs: teamai workflow start <name> --inputs '<json>', checked before the run exists (invalid_inputs). Every declared input is required.
    • Expressions only read inputs and steps. The falsy values are false, null, 0 and ''. == is strict and case-sensitive, and a missing path is null. They can only read steps the current step transitively needs.
    • Templates are only {{ inputs.* }} and {{ steps.* }}, plus {{ with.* }} on a prompt step that declares with; any other {{ }} in a prompt stays as written, and so does ${{ }}. A with value that is a single template keeps its JSON type. env values are strings. A template inside command fails pull and points to env:.
    • Endings. end finishes the innermost workflow. An inline workflow can declare ends and outputs. done also reports the end a top-level run reached, so CI can tell how it finished.
    • For [feat] Add bounded loops and parallel for_each #1079: sub-workflow steps are created when their parent becomes ready and are addressed by path (check.classify, with room for [n]). Leases stay one at a time.
    • For [feat] Publish privacy-preserving workflow run records and stats #1083: the run settles in one place. If feat(workflow): publish privacy-preserving run records to teamai-reports (#1083) #1126 merges first, I'll adapt the records here (path lookup, skipped, end).
    • Conformance gains scenarios for skipped steps, env, sub-workflow paths and recovery.
  9. SaulMoro
    self-assigned this
    on Oct 14, 2026

  10. SaulMoro commented on Oct 14, 2026

    @SaulMoro
    SaulMoro
    on Oct 14, 2026
    Collaborator

    Thanks @hsuchifeng for #1128. I went through it against the plan for this issue, and it caught gaps that I have now folded into #1129.

    Taken from #1128

    When a workflow step has end, it finishes the parent with that end once the child succeeds. My end-only check would have finished it without running the child.

    - id: triage
  workflow: triage-issue
  end: ready        # runs triage-issue first; the parent ends `ready` only if it succeeds

    Pull validates outputs, including fields against the step's output. The issue asks for it and my plan had dropped it.

    outputs:
  kind: steps.clasify.outputs.kind   # typo: pull fails instead of the parent always reading null

    inputs.* reads in if and templates are checked against the declared inputs.

    inputs: { issue: number }
...
  if: inputs.isue == 42              # typo: pull fails instead of the step being skipped silently

    env values written as YAML numbers or booleans are accepted and passed as text.

    env: { RETRIES: 3, DRY_RUN: true }   # the command gets RETRIES="3", DRY_RUN="true"

    Templates nested inside with objects and arrays are rendered, each string by the same rule.

    with:
  task: { title: "{{ inputs.title }}", files: [] }   # task = { title: "Fix login", files: [] }

    with is no longer silently ignored on a step that is not a workflow step. Following #818, I went one step further:

    • on a prompt step, with names values for that prompt, so one prompt file can serve several workflows;
    • on a command step, with fails pull and points to env:.
    - id: review
  prompt_file: prompts/review.md                     # the file uses {{ with.diff }}
  with: { diff: "{{ steps.diff.outputs.text }}" }

    Where #1129 differs, and why

    A failed step does not stop the run. Independent branches keep settling, and the run fails at the end unless it reached an end.

    - id: test
  command: npm test       # fails
- id: docs
  needs: []
  command: npm run docs   # #1128: never runs. #1129: runs, then the run ends failed

    Sub-workflows are snapshotted with the parent at start. A pull during a run cannot change or remove a child halfway through.

    t0  start ship                    → #1129 snapshots ship + check-issue v1
t1  pull brings check-issue v2
t2  ship reaches its `check` step → #1129 runs v1; #1128 loads v2, or fails if v2 is invalid

    Every step after the first declares needs, and needs: [] declares another root. When implicit order is mixed with roots, merge can run after a failed test as soon as any step declares needs.

    - id: test
  command: npm test
- id: notify
  needs: [test]
  if: steps.test.outcome == 'failed'
  command: ./notify.sh
- id: merge               # #1128: a root, so it merges after a failed test. #1129: fails pull until it declares needs
  command: gh pr merge

    Only {{ inputs.* }} and {{ steps.* }} are templates, plus {{ with.* }} on a prompt step that declares with. Any other {{ }}, and anything written as ${{ }}, stays literal, so prompts that quote GitHub Actions or Jinja keep working. A template inside command fails pull and points to env:.

    Add `token: ${{ secrets.GH_TOKEN }}` to the job for issue {{ inputs.issue }}.
→ #1129: "...${{ secrets.GH_TOKEN }}... issue 42"   #1128: "...token: $..."

    teamai workflow start <name> --inputs '<json>'. The inputs are checked before the run exists, so a top-level workflow with inputs can run from the CLI.

    teamai workflow start ship --inputs '{"issue":"42"}'
→ {"error":{"code":"invalid_inputs","message":"inputs.issue must be a number"}}   exit 1, no run created
  11. jeff-r2026
    closed this as completedin #1129on Oct 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions