Skip to content

feat(analytics): classify WriteError hierarchy out of unknown#120

Open
awkoy wants to merge 1 commit into
mainfrom
chore/classify-write-errors
Open

feat(analytics): classify WriteError hierarchy out of unknown#120
awkoy wants to merge 1 commit into
mainfrom
chore/classify-write-errors

Conversation

@awkoy

@awkoy awkoy commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The gap: write tool failures all landed in error_kind=unknown in BI. Every WriteError subclass (validation, backend, auth-denied, batch limits, unknown-op) was missing from _ERROR_KIND_TABLE in analytics/wrappers.py.
  • The reason tool_args_invalid didn't catch it: writes/dispatch.py catches pydantic.ValidationError at every op.pydantic_model.model_validate(...) call and re-raises as ValidationFailedError, which extends Exception directly (not pydantic.ValidationError). So the pydantic row in the table never fired on the write path.
  • The fix: add the 6 WriteError subclasses to the classifier (subclass-before-parent ordering preserved) plus a WriteError catch-all row. Bucket names mirror the stable ErrorCode literals already declared in writes/errors.py so BI dashboards can join the analytics event against the tool-result envelope by the same enum.

This is a direct follow-up to PR #118 — discovered while verifying that 0.1.3 actually shrinks the 44% unknown share. Triggered 6 real failure paths through live MCP tool calls; 3 of them (every write-tool path) were misrouted to unknown.

Test plan

  • make check clean — 508 tests pass, ruff format clean, mypy strict clean
  • 8 new parametrized cases in test_analytics_wrappers.py cover every WriteError subclass + the catch-all row
  • Existing tool_args_invalid row preserved with updated comment explaining why it doesn't fire on the write path
  • After merge: re-run the same 6-case verification and confirm write failures now route to write_validation_failed / write_backend_error instead of unknown

🤖 Generated with Claude Code

The `write` tool's structured error hierarchy (writes/errors.py) was never
listed in `_ERROR_KIND_TABLE`, so every write failure — including ones we
expected to surface as `tool_args_invalid` — bucketed to `unknown`.

The dispatcher catches `pydantic.ValidationError` at every `model_validate`
call site and re-raises as `ValidationFailedError` (a `WriteError`, NOT a
pydantic subclass), so the pydantic row in the table never fired on the
write path. This silently swallowed the bulk of write-tool failure
diagnostics in the 44% unknown bucket from PR #118.

Bucket names mirror the stable `ErrorCode` literals declared in
writes/errors.py so BI can join the analytics event against the tool-
result envelope using the same enum.

Verified live: re-running the same six failure cases from the previous
verification round now routes each write failure to its own bucket
instead of `unknown`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant