Establish A Reproducible Research Workflow Harness

[PSchmiedmayer]

Use Case

A researcher should be able to select a guided workflow for a common analysis task without designing the agent prompt.
The same project must remain usable through an open conversation when no workflow fits.

Problem

Heartwood has research Skills, specialists, task tracking, reviewed actions, and reproducible sessions, but no shared research-workflow definition.
The model currently derives stages, evidence, and stopping criteria from the prompt, making common analyses harder to reproduce, benchmark, and resume.

Solution

• Define a versioned research-workflow contract with declared inputs, stages, expected artifacts, evidence requirements, reviewer gates, budgets, and completion criteria.
• Execute the contract through OpenHands tasks and structured outcomes while the gateway owns state, recovery, policy, and interface projection.
• Pass stage-specific context and artifact references between tasks, and run deterministic checks before model review where possible.
• Ship three first-party workflows:
  • Dataset Readiness Review for schema, metadata, missingness, validity, duplicates, consistency, balance, and leakage risks.
  • Reproducible Baseline Analysis for the question, estimand, assumptions, data split, baseline, diagnostics, sensitivity checks, and report.
  • Independent Result Verification for environment reconstruction, re-execution, artifact comparison, discrepancies, and reproduction status.
• Reference existing verified Skills and specialist identifiers rather than introducing another catalog or distribution mechanism.
• Expose workflow selection, progress, evidence, recovery, and artifacts in every interface.

Acceptance Criteria

• Each bundled workflow completes a deterministic synthetic fixture and produces its declared artifacts and evidence.
• A stage cannot report completion when required evidence is missing or a required deterministic check failed.
• Pause, resume, cancellation, restart, and retry preserve one workflow sequence without repeating known work.
• Terminal, browser, and notebook clients show the same stage, task, evidence, review, and terminal state.
• The same workflow can run sequentially on a constrained route and use qualified parallel advisory work without changing its definition.
• Workflow outputs link to the experiment-run records from Record Reproducible Experiment Runs #125 and are measurable by the benchmark in Establish Model And Action-Risk Evaluation Benchmarks #44.
• Open conversation remains available and does not require a workflow selection.

Alternatives considered

Prompt-only workflow templates do not provide typed state, evidence gates, recovery, or comparable results.
A separate orchestration service would duplicate OpenHands tasks and Heartwood gateway behavior.
Starting with an OMOP- or Terra-specific workflow would make the foundation difficult to reuse across research domains.

Additional context

Issue #122 supplies bounded review and correction, and #124 supplies qualified parallel specialists.
Dataset-specific workflows such as #43 can build on this contract after the general research paths are stable.
The initial fixtures remain synthetic and do not establish scientific or institutional approval.

Code of Conduct

• I agree to follow this project's Code of Conduct and Contributing Guidelines