Review agent: detect backward-incompatible workflow input schema changes

[fullsend-ai-retro]

What happened

PR #1278 added a fullsend_ai_ref input to 6 reusable workflows with required: true and no default. This would have broken all existing deployed callers (per-org thin callers and per-repo shims) when the v0 tag advanced. The review agent ran 5 times on this PR. The first 3 runs flagged only protected-path and docs-currency issues. The 4th run finally caught the correctness issue. Meanwhile, human reviewer ralphbean and a 7-agent review squad both caught it independently on May 29. The author fixed it by changing to required: false with default: v0.

What could go better

The review agent lacks a specific heuristic for detecting backward-incompatible changes to workflow_call input schemas. Adding required: true without a default to an existing reusable workflow is a well-known breaking-change pattern in GitHub Actions. The review agent's general correctness analysis caught it eventually, but inconsistently — 3 runs missed it. A targeted check would catch this deterministically on the first run.

Existing issues #1069 (behavioral-change detection for workflow files) and #1275 (trace workflow_call chains) are related but neither specifically addresses input schema backward compatibility. #1778 covers breaking API changes in dependency bumps, not workflow inputs.

Confidence: Medium-high. I'm confident the review agent missed the issue on early runs and that a targeted check would help. I'm less certain whether the issue was present in the earliest commit (SHA 07c33c6) or was introduced later — the review runs were on different SHAs.

Proposed change

Add a backward-compatibility check to the review agent's correctness sub-agent (or as a dedicated workflow-specific check) that detects when a workflow_call or workflow_dispatch input is changed from required: false (or newly added) to required: true without a default value. This check should:

1. Parse workflow YAML files in the diff for on.workflow_call.inputs and on.workflow_dispatch.inputs sections.
2. Compare against the base branch to detect newly-required inputs without defaults.
3. Flag as HIGH or CRITICAL severity with a message explaining the backward-compatibility impact on existing callers.

This belongs in the review agent's skill or sub-agent definition in fullsend-ai/fullsend (likely the correctness sub-agent or a workflow-specific review check).

Validation criteria

On the next PR that adds a required: true workflow input without a default (or changes an existing input from optional to required), the review agent should flag it as a breaking change on the first review run. Test by submitting a PR with this pattern and verifying the review output includes a backward-compatibility finding.

---

Generated by retro agent from #1278

[fullsend-ai-triage]

Triage Summary

Severity: Medium | Category: Feature

Problem

The review agent lacks a targeted heuristic for detecting backward-incompatible changes to workflow_call/workflow_dispatch input schemas. On PR #1278, 3 of 4 review runs missed the breaking change (adding required: true without a default), catching it only on the 4th attempt via general correctness analysis.

Root Cause Hypothesis

The correctness sub-agent relies on general-purpose analysis rather than a specific check for this well-known GitHub Actions breaking-change pattern.

Recommended Fix

Add a backward-compatibility check to the review agent that:

1. Parses workflow YAML diffs for on.workflow_call.inputs and on.workflow_dispatch.inputs
2. Compares against the base branch to detect newly-required inputs without defaults
3. Flags as HIGH severity with an explanation of the impact on existing callers

Related issues #1069 (behavioral-change detection) and #1275 (trace workflow_call chains) cover adjacent concerns and could inform the implementation.

Proposed Test Case

Scenario: Detect required input without default
  Given a reusable workflow with an optional input
  When a PR changes that input to required: true with no default
  Then the review agent flags it as a breaking change on the first run

Scenario: Allow required input with default (non-breaking)
  Given a reusable workflow with an optional input
  When a PR changes that input to required: true with a default value
  Then the review agent does NOT flag a backward-compatibility issue

Edge cases:
  - New input added with required: true and no default
  - Existing required input has its default removed
  - Multiple inputs changed in the same workflow file

---

Labels: Retro-generated feature request targeting the review agent's correctness sub-agent in the harness, with medium priority given inconsistent but not total failure.

[fullsend-ai-prioritize]

RICE Priority Score: 1.2

Score breakdown

Dimension	Score	Reasoning
Reach	1	This affects users who maintain reusable workflows and their downstream callers within the platform. While breaking workflow input changes can cascade broadly, the specific pattern (adding required inputs without defaults) occurs infrequently — it's a subset of all workflow PRs. Scored as moderate reach affecting a meaningful but not universal set of users.
Impact	1.5	When the review agent misses this pattern, it can allow breaking changes to ship that break all downstream callers of reusable workflows. The PR #1278 demonstrated real risk — 3 of 4 review runs missed it. However, human reviewers and the 7-agent squad caught it independently, so there are backup layers. A deterministic first-run catch would be a significant reliability improvement.
Confidence	0.8	High confidence. The issue is extremely well-described with a concrete example (PR #1278), clear reproduction pattern, specific proposed implementation, and detailed validation criteria. The scope is well-bounded — parse workflow YAML diffs and compare input schemas against the base branch.
Effort	1	Medium effort. Requires parsing workflow YAML files for input schema sections, diffing against the base branch, and integrating the check into the correctness sub-agent. The logic itself is straightforward (detect required:true without default on new/changed inputs), but it touches the review agent's skill definitions and needs test coverage for edge cases.

Formula: (1 x 1.5 x 0.8) / 1 = 1.2