Skip to content

[BUG] SequentialWorkflow drift detection can rerun indefinitely on persistent low scores #1536

@shaun0927

Description

@shaun0927

Describe the bug
SequentialWorkflow._run_drift_detection() reruns the pipeline in a while True loop until the drift score crosses the threshold. If the drift score stays low, the workflow keeps rerunning without any retry ceiling.

That can burn tokens indefinitely and prevents the workflow from terminating in a bounded way when the judge never improves.

To Reproduce

  1. Create a SequentialWorkflow with drift_detection=True.
  2. Mock the drift agent so it always returns a score below the threshold.
  3. Mock agent_rearrange.run() to return any string.
  4. Call run().

At HEAD the loop is:

while True:
    ...
    if score >= self.drift_threshold:
        break
    result = self.agent_rearrange.run(**run_kwargs)

with no retry budget or timeout.

Expected behavior
Drift-triggered reruns should stop after a bounded number of attempts (or an explicit time budget) and return the latest result or raise a controlled failure.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions