Skip to content

docs: report only blocking errors for dropped items, not repaired fields - #9

Merged
metalwarrior665 merged 4 commits into
masterfrom
claude/push-data-schema-repair-u6072g
Jul 27, 2026
Merged

docs: report only blocking errors for dropped items, not repaired fields#9
metalwarrior665 merged 4 commits into
masterfrom
claude/push-data-schema-repair-u6072g

Conversation

@metalwarrior665

Copy link
Copy Markdown
Member

Summary

This PR fixes a critical misreporting bug where dropped items would list all validation errors encountered during repair attempts, including fields that were successfully fixed. Now only the errors that actually blocked the item are reported, making it much easier to diagnose why an item was dropped.

Key Changes

  • Separate blocking errors from repaired fields: Modified cleanItemFields() to return both the cleaned item and a blockingErrors array containing only the errors that prevented repair, not the ones that were successfully fixed.

  • Plan placeholder fixes holistically: Added planPlaceholderFixes() function that considers all errors on a path together before deciding on a placeholder value. This prevents composite keywords (like anyOf) from condemning a field that a sibling error could fix, and ensures null wins among competing placeholder options.

  • Enhanced error descriptions: Updated describeKeyword() to include the expected type for type errors (e.g., type number instead of just type), providing clearer explanations for why a field couldn't be placeholdered.

  • Improved logging: When no usable errors are reported, the log now says "the API reported no usable errors for" instead of listing an empty field set.

  • Updated documentation: Added comprehensive examples in README showing how to interpret dropped item reports, with a real-world Google Maps case demonstrating the fix.

Implementation Details

  • The fix handles the real-world case where a batch fails on 40+ required fields but only a couple are truly hopeless (e.g., required number and boolean fields that can't be placeholdered).
  • All errors on a placeholder path are evaluated together to avoid false positives from composite keywords.
  • null is preferred as a placeholder value when multiple fixes are available, as it's the least committal option.
  • Extensive test coverage added for mixed-type schemas, nullable unions, and proper error reporting.

https://claude.ai/code/session_01WGk7vrUxB7dAt64mLKyH7J

claude added 2 commits July 27, 2026 15:10
The release automation writes CHANGELOG.md without a trailing newline and
with a stray blank line, which fails `format:check` and blocks the
pre-commit hook on every subsequent change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGk7vrUxB7dAt64mLKyH7J
…locked a dropped item

A dropped item was reported with every validation error the API had raised
for it, including the ones we had just successfully repaired. On a schema
with 40+ required fields that made the log claim `/title (type)`,
`/url (type)` and `/categories (type)` were "unfixable" when in fact they
had been placeholdered to '' and [], and only the required `number` /
`boolean` fields were hopeless. The repair behaviour was correct; the report
sent you after the wrong fields.

- Collect every blocking error instead of bailing on the first one, so a
  dropped item names all the fields standing in its way, and report only
  those in the log and in `droppedItems[].errors`.
- Consider all errors on a placeholder path together. AJV reports composite
  keywords next to the branch errors that explain them, so a nullable object
  arrives as `type: object`, `type: null` and `anyOf` at once; judging them
  one at a time let the trailing `anyOf` condemn a field its sibling had
  just fixed, and let a later branch overwrite an earlier branch's value. A
  path is now only a blocker when none of its errors offers a fix, and
  `null` wins between competing fixes.
- Include the expected type in `type` labels (`/imagesCount (type number)`),
  which for a dropped field is the whole explanation.
- Drop the dangling empty field list when the API reports an invalid item
  with no errors at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGk7vrUxB7dAt64mLKyH7J
@metalwarrior665 metalwarrior665 changed the title Report only blocking errors for dropped items, not repaired fields docs: report only blocking errors for dropped items, not repaired fields Jul 27, 2026
claude added 2 commits July 27, 2026 15:21
For a manual beta release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGk7vrUxB7dAt64mLKyH7J
…he field isn't lost from the drop report

A required field the caller got wrong itself went unnamed when the item was
dropped. `permanentlyClosed: 'rwer'` against a required `boolean` was deleted
in round 1 as bad user data, which threw away the one thing that error taught
us — the schema wants a boolean there. Round 2 then saw only `required`, which
looks repairable, so the field was placeholdered to `null` and left out of the
drop report; only round 3 would have identified it, and the item was already
gone by then.

Deleting a field for failing a `type` we have no placeholder for now records
that type against the path. A later `required` error on the same path
recognises the dead end, reports it alongside the other blockers, and drops the
item on that round instead of the one after. Only `type` is carried over: the
expected type is fixed by the schema, whereas `pattern` / `minLength` / `enum`
say nothing certain about whether our `''` would satisfy them.

For the reported schema the second round now reads:

    dropped 1 item(s) on unfixable fields: /imagesCount (type number),
    /isAdvertisement (type boolean), /permanentlyClosed (type boolean),
    /temporarilyClosed (type boolean)

An item whose only fault is one such field also drops a round earlier, in two
pushes rather than three.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WGk7vrUxB7dAt64mLKyH7J
@metalwarrior665
metalwarrior665 merged commit f1fd2f3 into master Jul 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants