Review agent should detect env.sandbox references unsatisfied by calling workflows

[fullsend-ai-retro]

What happened

On PR #2758, the review agent ran 4 times and approved the final commit. It correctly identified test-integrity issues (weakened assertions) but missed a cross-file consistency bug: env.sandbox in harness/review.yaml references ${REVIEW_FINDING_SEVERITY_THRESHOLD}, but the reusable workflow that invokes the harness does not define this variable. The Qodo code review bot caught this; the fullsend review agent did not. The ADR 0055 migration will repeat this exact pattern for 5 more agents (triage, code, fix, retro, prioritize), making this a recurring risk.

What could go better

The review agent performed well on code-level analysis (test assertions) but lacked a check for cross-file consistency between harness configs and their calling workflows. This class of bug — where a config references a variable that no caller provides — is particularly important during migration PRs that change the variable delivery mechanism. Confidence is high that this is a real gap: the agent had access to both files and could have detected the mismatch. Existing issues #2676 (incomplete migrations) and #2703 (env var validation before LLM start) are adjacent but do not cover review-agent detection of env.sandbox/workflow mismatches specifically.

Proposed change

Add a review dimension or checklist item to the review agent's pr-review skill (or its sub-agent definitions) for harness/workflow consistency checks. When a PR modifies env.sandbox or env.runner sections in harness YAML files, the review agent should:

1. Identify all ${VAR} references in the modified env sections.
2. Check whether each referenced variable is defined in the corresponding reusable workflow's env: block for the agent execution step.
3. Flag any variable that is referenced but not provided by the workflow.

This could be implemented as guidance in skills/pr-review/SKILL.md or as a dedicated sub-agent for harness-config reviews. The specific file to update depends on the review agent's architecture, but the skill definition at internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md is a natural starting point.

Validation criteria

On the next PR that migrates a harness from host_files/runner_env to env.sandbox (e.g., triage or code agent migration under ADR 0055), the review agent should flag any ${VAR} references in env.sandbox that are not defined in the corresponding reusable workflow. Validate by checking the review agent's findings on at least 2 such migration PRs.

---

Generated by retro agent from #2758

[fullsend-ai-triage]

🤖 Finished Triage · ✅ Success · Started 3:15 PM UTC · Completed 3:18 PM UTC
Commit: 9f4f2b6 · View workflow run →

[fullsend-ai-triage]

Triage Summary

This is a well-defined feature request to add harness/workflow variable consistency checking to the review agent.

Problem

The review agent approved PR #2758 despite a cross-file consistency bug: env.sandbox in harness/review.yaml referenced ${REVIEW_FINDING_SEVERITY_THRESHOLD}, but the calling reusable workflow did not define it. The Qodo bot caught this; the fullsend review agent did not.

Why it matters

The ADR 0055 migration will repeat this pattern for 5 more agents. Without this check, each migration PR risks the same class of runtime failure.

Related issues

• Fix: reusable-review.yml missing REVIEW_FINDING_SEVERITY_THRESHOLD env var #2935 — Tracks the actual bug fix for the missing env var (complementary, not duplicate)
• Review agent should detect incomplete mechanical migrations across docs and examples #2676 — Incomplete migration detection (adjacent but different scope)
• Harness should validate agent definition and critical env vars before starting LLM conversation #2703 — Env var validation before LLM start (runtime validation, not review-time)

Recommended approach

Add a review dimension or checklist item to the pr-review skill (internal/scaffold/fullsend-repo/skills/pr-review/SKILL.md) or its sub-agent definitions. When a PR modifies env.sandbox or env.runner sections, the review agent should:

1. Extract all ${VAR} references from modified env sections
2. Verify each is defined in the corresponding reusable workflow's env: block
3. Flag unresolved references as a finding

Proposed test case

Scenario: Unsatisfied env.sandbox variable reference
  Given a harness YAML with env.sandbox referencing ${SOME_VAR}
  And the corresponding reusable workflow does not define SOME_VAR
  Then the review agent should flag the unresolved reference

Scenario: All variables satisfied
  Given a harness YAML with env.sandbox referencing ${SOME_VAR}
  And the corresponding reusable workflow defines SOME_VAR
  Then no finding should be raised

Edge cases:
  - Variables with shell defaults: ${VAR:-default}
  - Variables defined in parent but not direct caller workflow
  - Variables removed from workflow but still referenced in harness

---

Labels: Review agent feature request involving harness config consistency checks, with medium priority due to 5 upcoming migration PRs at risk.