Skip to content

Commit b46b157

Browse files
committed
fix(#22): enforce structured output in code agent
The code agent was missing structured output enforcement, causing validation_loop failures. The review and fix agents both have explicit structured output sections and mandatory final validation steps, but the code agent had neither. Changes: - agents/code.md: add "Structured output" section documenting the requirement to produce code-result.json and validate it with fullsend-check-output before exiting. Update failure handling to reference structured output in the handoff contract. - skills/code-implementation/SKILL.md: add step 11 as a mandatory final step to validate structured output against the schema. Add forward reference from step 3 (where the file is first written) to step 11. Update progress markers list. Require structured output even for partial work exits. Closes #22 Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
1 parent aa64fc8 commit b46b157

2 files changed

Lines changed: 69 additions & 8 deletions

File tree

agents/code.md

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -92,17 +92,38 @@ the review agent — if the triage was wrong, your code will fail review.
9292
- If the retry limit is exceeded and tests still fail, do not commit broken
9393
code. Stop. The post-script reports the failure.
9494

95+
## Structured output
96+
97+
You MUST produce a JSON file at `$FULLSEND_OUTPUT_DIR/$FULLSEND_OUTPUT_FILE`
98+
(typically `code-result.json`) that documents the target branch for PR
99+
creation. The `code-implementation` skill describes the schema and the exact
100+
step where you write it. The post-script reads this file to determine which
101+
branch to target the PR against. Without this file, the validation loop
102+
rejects the run and retries.
103+
104+
After writing the file, validate it before exiting:
105+
106+
```bash
107+
fullsend-check-output "${FULLSEND_OUTPUT_DIR}/${FULLSEND_OUTPUT_FILE}"
108+
```
109+
110+
If validation fails, read the error output, fix the JSON file, and
111+
re-run the check. If it still fails after 3 attempts, write the best
112+
JSON you have and exit.
113+
95114
## Failure handling
96115

97116
Secret scanning is **non-negotiable**. The `scan-secrets` helper runs before
98117
tests on every verification pass. If secrets are detected — or if the helper
99118
script is missing — hard stop. Do not improvise a replacement or skip the scan.
100119

101120
Your exit state is the handoff contract:
102-
- **Clean commit on the feature branch** → the post-script pushes and creates
103-
the PR (after its own authoritative secret scan).
121+
- **Clean commit on the feature branch + valid structured output** → the
122+
post-script pushes and creates the PR (after its own authoritative secret
123+
scan).
104124
- **No commit** → the post-script reads your transcript and exit code to
105-
report the failure.
125+
report the failure. Structured output should still be written when possible
126+
so the post-script knows which branch was targeted.
106127

107128
## Detailed implementation procedure
108129

skills/code-implementation/SKILL.md

Lines changed: 45 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ echo "::notice::STEP <N>: <title>"
5656
```
5757

5858
This uses GitHub Actions annotation syntax so it surfaces in the run
59-
summary. **Do this at steps 1, 3, 5, 9a, 9b, 9c, and 10.**
59+
summary. **Do this at steps 1, 3, 5, 9a, 9b, 9c, 10, and 11.**
6060

6161
## Time budget
6262

@@ -186,8 +186,8 @@ use that branch. Otherwise, determine the repo's default branch:
186186
git rev-parse --abbrev-ref origin/HEAD | cut -d/ -f2
187187
```
188188

189-
Write your chosen branch to the structured output file so the post-script
190-
knows which branch to target the PR against:
189+
Write your chosen branch to the structured output file **early** so
190+
it is available even if the agent hits a timeout or error later:
191191

192192
```bash
193193
mkdir -p "${FULLSEND_OUTPUT_DIR}"
@@ -198,9 +198,8 @@ cat > "${FULLSEND_OUTPUT_DIR}/${FULLSEND_OUTPUT_FILE}" <<RESULT
198198
RESULT
199199
```
200200

201-
Write this output early (during planning, after determining the target
202-
branch) so it is available even if the agent hits a timeout or error later.
203201
The post-script validates this against the repo's allowed branches.
202+
You will re-validate this file in the final step (step 11).
204203

205204
### 4. Check for existing branch
206205

@@ -739,6 +738,43 @@ message. The post-script runs an authoritative pre-commit on the runner.
739738
**Do not push the branch.** The post-script handles pushing, PR creation,
740739
and failure reporting.
741740

741+
### 11. Validate structured output
742+
743+
**This step is MANDATORY.** The harness runs a validation loop that
744+
checks `$FULLSEND_OUTPUT_DIR/$FULLSEND_OUTPUT_FILE` against
745+
`schemas/code-result.schema.json`. If validation fails, the harness
746+
retries the agent. Producing a valid output file is not optional.
747+
748+
You wrote the initial output file in step 3. Confirm it still exists
749+
and contains the correct target branch:
750+
751+
```bash
752+
echo "::notice::STEP 11: Validate structured output"
753+
cat "${FULLSEND_OUTPUT_DIR}/${FULLSEND_OUTPUT_FILE}"
754+
```
755+
756+
The file must be valid JSON with exactly one field:
757+
758+
```json
759+
{
760+
"target_branch": "main"
761+
}
762+
```
763+
764+
**Schema compliance:** The schema uses `additionalProperties: false`.
765+
Only the `target_branch` field is allowed. Any extra fields will cause
766+
validation to fail.
767+
768+
Validate the output against the schema:
769+
770+
```bash
771+
fullsend-check-output "${FULLSEND_OUTPUT_DIR}/${FULLSEND_OUTPUT_FILE}"
772+
```
773+
774+
If validation fails, read the error output, fix the JSON file, and
775+
re-run the check. If it still fails after 3 attempts, write the best
776+
JSON you have and exit.
777+
742778
## Partial work
743779

744780
If you hit a token limit or context window boundary before completing the
@@ -748,6 +784,10 @@ code is caught at the review stage, not the implementation stage. The commit
748784
message should note that the work is partial (e.g., "partial implementation"
749785
in the description) so the review agent and post-script can act accordingly.
750786

787+
**Structured output is still required for partial work.** The output file
788+
written in step 3 must exist and be valid. Run `fullsend-check-output`
789+
(step 11) even when exiting early.
790+
751791
## Constraints
752792

753793
The agent definition (`agents/code.md`) is the authoritative list of

0 commit comments

Comments
 (0)