Skip to content

Scheduler: same-lane dependency costs a full wave barrier — warnings/docs now, lane-local topological ordering later #641

Description

@HenryLach

Problem

A dependency between two tasks that share a file-scope lane costs a full wave barrier. computeWaves (extensions/taskplane/waves.ts:216-262, Kahn) counts every pending-target edge toward in-degree (:224-230) and has no notion of lanes — lanes are assigned after waves (:1549-1555). So TP-2101 depends on TP-2100 promotes TP-2101 to wave 2, where it waits for every unrelated wave-1 lane to finish and merge, even though the two tasks would run back-to-back on one lane anyway.

Observed on a six-packet penster batch; the shape ("B builds on A, and they touch the same files") is the common case for dependencies, not an exotic one.

This is a feature request, not a bug — the scheduler does what it specifies.

What the executor already guarantees (undocumented, load-bearing)

Within one affinity lane, tasks run sequentially in a single worktree, each starting on top of the previous task's commits:

  • executeLaneV2 iterates lane.tasks in order — execution.ts:3047.
  • After success: commitTaskArtifacts (:564-600, git add -A + commit), then only checkout -- . && clean -fd (:3164-3170) — uncommitted noise is dropped, commits stay. No reset --hard to base exists.
  • Worktrees are created once per wave from the orch branch (execution.ts:2047-2054worktree.ts:323-333); merge is per lane branch at wave end.
  • A failed/stalled task sets shouldSkipRemaining and the rest of the lane is skipped (:3049-3067, :3172-3174); a held task (Cap-ruling hold: runtime must block .DONE and merge while an operator ruling is pending (.PENDING-RULING) #627) leaves the rest pending.
  • Lane order = affinity-group ID sort (waves.ts:411) → assignGroupToLane (:902-913) → allocateLanes preserves order (:1458-1462).

Same lane vs. separate waves:

Same lane, back-to-back Separate waves
B starts from A's commits on the lane branch orch branch after wave 1 merged
Between A and B seconds wave merge (agent, conflicts, tests) + re-allocation; can fail
Who waits nobody else every wave-1 lane
B sees other lanes' work no yes

Same-lane is right when B depends only on A; separate waves is right when B depends on everything in the earlier wave. Today the scheduler cannot tell them apart and always pays the expensive one.

Workaround available today (validated against source)

Drop the **Task:** edge; make sure the two packets' ## File Scope overlap (so applyFileScopeAffinity, waves.ts:350-417, unions them; ID order then guarantees A before B); give B a fail-closed Step 0 preflight that checks A's artifacts (not its .DONE). Cost: the runtime loses the dependency bookkeeping (transitive skip-dependents) — covered in practice by the same-lane skip.

There is no existing mechanism to express "B after A, same lane, same wave": the dependency parser extracts only IDs (discovery.ts:665-692), orchestrator.dependencies has only source/cache (types.ts:40-41), and no lane-order hint exists anywhere.

Proposal — two parts, the second earns its way in

Part 1 — zero-risk: warnings + docs (ship first)

  1. Discovery / taskplane doctor warning when a dependency target shares an affinity group with the dependent: "TP-2101 depends on TP-2100 in the same file-scope lane; the edge costs a wave barrier. Same-lane ID ordering already runs 2100 before 2101 — consider dropping the edge and adding a Step 0 artifact preflight."
  2. Discovery warning when a task's ## File Scope includes a shared log/context file (e.g. CONTEXT.md) that collapses N tasks into one lane: "file scope includes a shared file — this serializes N tasks; intended?" (Today this is a convention only; the template puts CONTEXT.md under Context to Read First, not File Scope. Parallelism depends on nobody breaking it.)
  3. Docs: the executor guarantees and the same-lane/separate-waves table above in docs/explanation/architecture.md and docs/reference/task-format.md.

Part 2 — lane-local topological ordering

Satisfy a dependency whose target is in the same affinity group by in-lane order within the same wave; promote to a later wave only when the target is in a different group.

Size: moderate, concentrated in waves.ts (~150–250 lines + tests). Compute applyFileScopeAffinity over the whole pending set once (pure Union-Find, wave-independent); in computeWaves, an edge with find(dep) === find(node) does not count toward in-degree and is recorded as an intra-group constraint; assignGroupToLane orders each group by a topological sort of those constraints (ID tie-break) replacing group.sort() at :411. allocateLanes already preserves order; the executor needs no change. Fall back to wave promotion when the target is in a different repo (workspace mode) or when enforceGlobalLaneCap (:1010+) would split the group — make cap redistribution group-preserving.

Risk list (why Part 2 is sequenced):

  1. Resume/retry (gating). The wave is currently the unit of recovery. An in-lane dependent B skipped because A failed must be re-queued when A is retried (orch_retry_taskresetTaskSegmentsForRetry handles one task today). Tier-0 attemptWorkerCrashRetry builds a single-task retry lane — B would not re-run inside it. This re-queue must ship with Part 2, not after.
  2. Held lanes (Cap-ruling hold: runtime must block .DONE and merge while an operator ruling is pending (.PENDING-RULING) #627) — B sits pending behind a held A; hold-first resume re-executes A in place. Correct by construction; add the fixture.
  3. Completion authority / finalize — per-unit predicates, no wave/lane awareness. No exposure.
  4. Merge — B's commits on top of A's on one lane branch, identical to today's affinity groups. No exposure. Dashboard wave bars shift (cosmetic).
  5. Determinism tests — plan snapshots in orch-pure-functions / waves tests change. Churn, not risk.

Minimum proving fixture (extensions/tests/): tasks A and B with overlapping fileScope, B depends on A → (i) computeWaves yields one wave; (ii) both on one lane, A at order 0; (iii) a task C depending on A with disjoint scope still lands in wave 2; (iv) enforceGlobalLaneCap at maxLanes: 1 keeps A before B; (v) behavioural executeLaneV2 (mock spawnAgent, real git): A commits a file, B's mock asserts it sees it; A fails → B skipped; A retried → B re-queued.

Shipping: as a pre-release like #629/#631 — feature branch → Sage design + review → PR → local build → one dogfood batch on Taskplane itself of exactly the two-task shape before any multi-packet consumer batch. Sequence after the #627 hold path has had a live exercise (currently the larger untested surface).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions