What happened
On PR #12 in fullsend-ai/agents, the review agent's code-review skill flagged customer-research as a missing dependency 6 times (medium severity) because no skills/customer-research/ directory existed locally. The skill exists in fullsend-ai/.fullsend/customized/skills/customer-research/SKILL.md as an org-level customization. The review agent had no mechanism to check org-level or platform-level skill resolution before raising the finding.
What could go better
The review agent's correctness and intent-coherence sub-agents should understand the skill resolution chain when evaluating agent definition files. Before flagging a skill reference as missing, the agent should consider whether the skill could be resolved from org-level .fullsend customization or upstream platform skills. This is distinct from existing issues: #237 covers harness loading policy (not review validation), #2744 covers grepping before flagging upstream-downstream violations (different class), and #1013 covers deduplication across iterations (not false-positive prevention).
Proposed change
Add guidance to the code-review skill (or a new sub-agent prompt) that when evaluating agent definition files (files matching agents/*.md), skill references in frontmatter should be validated against the full resolution chain, not just local directories. The review agent could either: (a) check for documentation in the repo explaining skill resolution (like the AGENTS.md addition proposed above), or (b) be taught that skills: frontmatter entries combined with conditional prompt language ('if available') indicate optional cross-repo dependencies that are not bugs.
Validation criteria
Review agent should not produce medium-or-higher findings for missing local skill directories when the agent prompt uses conditional language for that skill. Validate on next 5 PRs across fullsend-ai repos that modify agent frontmatter skill declarations.
Generated by retro agent from fullsend-ai/agents#16
What happened
On PR #12 in fullsend-ai/agents, the review agent's code-review skill flagged
customer-researchas a missing dependency 6 times (medium severity) because noskills/customer-research/directory existed locally. The skill exists infullsend-ai/.fullsend/customized/skills/customer-research/SKILL.mdas an org-level customization. The review agent had no mechanism to check org-level or platform-level skill resolution before raising the finding.What could go better
The review agent's correctness and intent-coherence sub-agents should understand the skill resolution chain when evaluating agent definition files. Before flagging a skill reference as missing, the agent should consider whether the skill could be resolved from org-level
.fullsendcustomization or upstream platform skills. This is distinct from existing issues: #237 covers harness loading policy (not review validation), #2744 covers grepping before flagging upstream-downstream violations (different class), and #1013 covers deduplication across iterations (not false-positive prevention).Proposed change
Add guidance to the code-review skill (or a new sub-agent prompt) that when evaluating agent definition files (files matching
agents/*.md), skill references in frontmatter should be validated against the full resolution chain, not just local directories. The review agent could either: (a) check for documentation in the repo explaining skill resolution (like the AGENTS.md addition proposed above), or (b) be taught thatskills:frontmatter entries combined with conditional prompt language ('if available') indicate optional cross-repo dependencies that are not bugs.Validation criteria
Review agent should not produce medium-or-higher findings for missing local skill directories when the agent prompt uses conditional language for that skill. Validate on next 5 PRs across fullsend-ai repos that modify agent frontmatter skill declarations.
Generated by retro agent from fullsend-ai/agents#16