Skip to content

yama: add soundness/polarity lens + 2 blocking criteria (review follow-up to #56)#58

Merged
jagguji merged 2 commits into
mainfrom
yama-soundness-lens
Jun 15, 2026
Merged

yama: add soundness/polarity lens + 2 blocking criteria (review follow-up to #56)#58
jagguji merged 2 commits into
mainfrom
yama-soundness-lens

Conversation

@jagguji

@jagguji jagguji commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

What

Follow-up to #56 (merged before this feedback could be folded in). Applies the three review suggestions to yama.config.yaml so the AI reviewer is pointed at the one blind spot it currently has — an unsound type that compiles.

Why

The config (correctly) steers Yama away from re-running the mechanical CI gates, and blocks Obj.magic/@unwrap/%identity. But the highest-value bugs in our review history were none of those — they compiled green on every gate and needed a human:

As configured, the reviewer would likely miss these too. This PR closes that.

Changes (config-only — does not touch the generator)

  1. workflowInstructions — a "compilation is NOT soundness" lens (CI proves compiles + matches baseline, not soundness; a fake behind a shared typeRef doesn't bubble to the report) + a PRODUCE/RECEIVE polarity rule ('a is sound only in a consumer-supplied input).
  2. focusAreas (mapping correctness) — an all-cases-or-flag line: every union/overload variant must be reachable, or the whole prop is flagged — never silently dropped.
  3. blockingCriteria (4 → 6) — (a) an output-only 'a that doesn't round-trip; (b) a fake emitted UNFLAGGED inside a shared *Types.res entry.

Verified: all five additions present, blocking criteria count 4 → 6, indentation mirrors the existing entries. (No YAML linter available locally; structure matches the surrounding blocks exactly.)

…w-up to #56)

Applies the review feedback from PR #56 that merged before it could be folded
in. The Yama reviewer was steered toward 'no Obj.magic/@unwrap/%identity' but
not toward the subtler class of bug that every mechanical gate (golden compile,
bench) passes green on — an unsound type that compiles. That's exactly the #55
regression a human had to catch (an output-only 'a from an `any` getter return).

- workflowInstructions: a 'compilation is NOT soundness' lens (CI proves
  compiles + matches baseline, not soundness; a fake behind a shared typeRef
  doesn't bubble to the report) and a PRODUCE/RECEIVE polarity rule ('a is sound
  only in a consumer-supplied input position).
- focusAreas: an all-cases-or-flag line (every union/overload variant must be
  reachable or the whole prop is flagged — never silently dropped).
- blockingCriteria (+2 -> 6): output-only 'a that doesn't round-trip; a fake
  emitted UNFLAGGED inside a shared *Types.res entry.

Config-only; doesn't touch the generator.
@github-actions

github-actions Bot commented Jun 15, 2026

Copy link
Copy Markdown

Benchmark: ✅ PASS

Package Compile Diff vs baseline usable review broken Verdict
@juspay/blend-design-system@0.0.36 identical 94 0 0 ✅ PASS
react-day-picker@10.0.1 identical 27 0 0 ✅ PASS
react-tooltip@6.0.7 identical 1 0 0 ✅ PASS
react-markdown@10.1.0 identical 0 2 0 ✅ PASS
@smastrom/react-rating@1.5.0 identical 1 0 0 ✅ PASS
clsx@2.1.1 identical 0 0 0 ✅ PASS
hono@4.12.25 identical 0 0 0 ✅ PASS
@base-ui-components/react@1.0.0-rc.0 identical 195 0 0 ✅ PASS

@pkg-pr-new

pkg-pr-new Bot commented Jun 15, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@juspay/rescript-bindgen@58

commit: ef903fd

…ew note)

Per review on #58: the 'fake string inside a shared *Types.res entry' blocking
criterion is judgment-heavy — an LLM could flag legitimately-string fields. Add
an explicit guard: only BLOCK after VERIFYING against the source .d.ts that the
TS type was genuinely richer and got narrowed; a field that is legitimately
`string` in TypeScript is not a violation.
@jagguji

jagguji commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the thorough review — and for running the YAML parse I couldn't (good to have psych confirm the backticks/colons/apostrophes in the quoted strings are clean).

Applied your one actionable note: tightened the hidden-fake criterion to curb false positives. It now requires the reviewer to VERIFY against the original .d.ts that the TS type was genuinely richer (union/object/callback) and got narrowed to string before blocking — and says explicitly that a field legitimately typed string in TypeScript is not a violation. That should stop it BLOCKing on every honest string field.

On your other two notes — both intentional, leaving as-is:

  • The polarity idea appearing in 3 places is deliberate reinforcement for an LLM prompt (not accidental redundancy).
  • Agreed there's no mechanical gate that proves the reviewer will catch these — it's guidance, and the only real signal is whether the next soundness regression gets caught in review. The PR is honest about that.

Pushed as ef903fd.

@jagguji jagguji merged commit cc7f89c into main Jun 15, 2026
9 checks passed
@jagguji jagguji deleted the yama-soundness-lens branch June 15, 2026 15:12

@Tara-ag Tara-ag left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

File reviewed: yama.config.yaml (1 file, +27 lines)

New issues raised: 0

Assessment

This PR successfully adds critical soundness guidance to the Yama review configuration based on lessons from issues #55, #40, and #49. The changes are:

  1. CRITICAL LENS — Compilation is NOT soundness; CI gates prove compilation + baseline match, not type safety
  2. POLARITY'a is only sound in consumer-supplied INPUT positions (round-trips); OUTPUT-only 'a unifies with anything
  3. All-cases-or-flag — Every union/overload variant must be reachable; never silently drop cases
  4. Two new blocking criteria — Output-only 'a and unflagged fakes in shared types now block merges

Verification:

  • ✅ YAML structure valid with consistent 2-space/4-space indentation
  • ✅ Blocking criteria count: 4 → 6 (matches PR description)
  • ✅ All three historical issues (#55, #40, #49) addressed in guidance
  • ✅ Config-only change — no generator code modified

Decision: Approve. The tightened criteria will help catch unsound types that compile successfully, addressing the highest-value blind spot in automated review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants