Skip to content

fix(triage): degrade unusable supplied artifacts to null instead of failing - #25

Merged
niranjan94 merged 1 commit into
release/v2from
fix/triage-degrade-unusable-artifacts
Jun 16, 2026
Merged

fix(triage): degrade unusable supplied artifacts to null instead of failing#25
niranjan94 merged 1 commit into
release/v2from
fix/triage-degrade-unusable-artifacts

Conversation

@niranjan94

Copy link
Copy Markdown
Owner

Problem

A real triage run failed hard on konfirmity/app#562:

[{ "code": "custom", "message": "content is required when source='body'", "path": ["supplied_spec"] }]

The agent judged the issue body to be a spec (artifact-detection rule #3) and set supplied_spec.source = "body" but did not extract content. The schema's hard .refine then rejected the entire triage decision, escalating a recoverable gap into a fatal failure that requires manually removing the shopfloor:failed:triage label to retry.

Two things make this a bug rather than correct strictness:

  • The downstream apply step already no-ops on a missing field (apply.ts only seeds a spec PR when source === "body" && content).
  • The .refine never constrained the model — refinements do not survive zodToJsonSchema, so they only fired as a post-hoc .parse() throw.

Fix

  • decision.ts — replace the two .refines with a .transform that degrades an unusable artifact (body without content, or path without path) to null. Well-formed artifacts still seed PRs; malformed ones fall through to the normal stage flow. Verified that the transform is transparent to zodToJsonSchema for both the Claude (default) and Codex (openAi) targets — the generated output contract is unchanged.
  • prompt.system.md — state explicitly that content is required for source:"body" and path for source:"path", and that a missing required field is discarded, so the agent stops emitting the bad shape.
  • test/stages/triage.test.ts — regression tests: body-without-content and path-without-path both degrade to null and route through needs-spec rather than throwing.
  • dist/index.cjs — rebuilt (committed bundle).

Design note

Chose silent degradation over backfilling content from the issue body. #562 has no explicit ## Shopfloor Spec marker (judgment-based detection), so re-running the spec stage is the designed default for a large issue; backfilling the whole body would add magic with edge-case risk.

Verification

  • pnpm typecheck clean
  • pnpm build clean (bundle regenerated)
  • pnpm test -> 286 passed (incl. 2 new degradation tests)
  • biome check clean on changed files

…ailing

The triage decision schema rejected a supplied_spec/supplied_plan whose
source did not carry its required field (a "body" source without content,
or a "path" source without path), failing the entire triage run and
forcing a manual shopfloor:failed:triage label removal to retry.

Replace the two hard refinements with a transform that degrades an
unusable artifact to null, so triage completes and the stage runs
normally. The downstream apply step already no-ops on a missing
path/content, so the seeding optimization is the only thing skipped.

The refinements never constrained the model (they do not survive
zodToJsonSchema); they only fired as a post-hoc parse failure. The
transform is equally transparent to schema generation for both the
Claude and Codex adapters. The prompt is tightened to state that the
required field must be populated so the agent stops emitting the bad
shape.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shopfloor agent review: clean across 4/4 reviewers.

  • PR #25 follows all documented conventions in CLAUDE.md (committed dist bundle, correct prompt/test file placement, no forbidden patterns or formatting-rule violations); no compliance issues found.
  • The .refine→.transform swap correctly degrades unusable supplied artifacts to null; downstream apply already guards on path/content presence, the type export is preserved, and the new tests genuinely exercise the transform via the mock agent's schema.parse.
  • The diff is a Zod validation refactor (refine→transform) for triage artifacts plus prompt/test updates; it introduces no exploitable security patterns.
  • The triage decision refactor (.refine → .transform), prompt update, and regression tests are clean with no maintainability smells worth flagging.

@niranjan94
niranjan94 merged commit cc80a9f into release/v2 Jun 16, 2026
6 checks passed
@niranjan94
niranjan94 deleted the fix/triage-degrade-unusable-artifacts branch June 16, 2026 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant