Skip to content

Add Bounded Agent Review And Correction Workflows #122

Description

@PSchmiedmayer

Use Case

A researcher should be able to ask Heartwood to review a proposed or completed analysis, verify the important findings, and correct confirmed problems without manually prompting each pass.
The workflow must remain bounded, inspectable, and subject to the existing action policy.

Problem

Heartwood has advisory research specialists, but their results are unstructured and each review is started manually.
There is no shared finding model, independent verification step, duplicate suppression, or bounded review-and-correction loop.
Repeated model-driven review can therefore become noisy, expensive, or circular.

Solution

  • Define a typed review finding with category, severity, evidence, affected paths, verification state, and disposition.
  • Reuse OpenHands structured outputs, task outcomes, hooks, and specialist tasks for review instead of adding another agent runtime.
  • Run only the reviewers relevant to the task, verify candidate findings independently, and combine duplicates before presenting them.
  • Add a bounded review -> propose correction -> verify workflow with explicit model-step, token, cost, time, and retry limits.
  • Route every project mutation through the parent conversation and existing grouped approval flow.
  • Project review state through the gateway-owned workflow state used by the terminal, browser, and notebook bridge.

Acceptance Criteria

  • Deterministic tests cover valid, duplicate, unsupported, and false-positive findings.
  • A capable-model test finds seeded coding, statistical, and reproducibility problems in the Reproducible Baseline Analysis from Establish A Reproducible Research Workflow Harness #130 and verifies the approved correction.
  • The workflow stops at its configured limits and can be paused, resumed, steered, or cancelled.
  • Restart and command retry do not repeat a review, model call, correction, or tool action with a known outcome.
  • Review findings replay consistently across interfaces while audit exports remain content-minimized.
  • Benchmark thresholds from Establish Model And Action-Risk Evaluation Benchmarks #44 gate any claim of autonomous correction quality.

Alternatives considered

A separate reviewer service would duplicate OpenHands task and lifecycle behavior.
Unbounded self-correction and automatic approval were rejected because they provide no reliable stopping rule or safety boundary.

Additional context

The workflow harness in #130 decides when a review gate is required; this issue owns finding verification and bounded correction.
It may propose changes autonomously, but it does not bypass the configured action confirmation policy.

Code of Conduct

  • I agree to follow this project's Code of Conduct and Contributing Guidelines

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    • Status
      Backlog

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions