feat(analytics): classify WriteError hierarchy out of unknown#120
Open
awkoy wants to merge 1 commit into
Open
Conversation
The `write` tool's structured error hierarchy (writes/errors.py) was never listed in `_ERROR_KIND_TABLE`, so every write failure — including ones we expected to surface as `tool_args_invalid` — bucketed to `unknown`. The dispatcher catches `pydantic.ValidationError` at every `model_validate` call site and re-raises as `ValidationFailedError` (a `WriteError`, NOT a pydantic subclass), so the pydantic row in the table never fired on the write path. This silently swallowed the bulk of write-tool failure diagnostics in the 44% unknown bucket from PR #118. Bucket names mirror the stable `ErrorCode` literals declared in writes/errors.py so BI can join the analytics event against the tool- result envelope using the same enum. Verified live: re-running the same six failure cases from the previous verification round now routes each write failure to its own bucket instead of `unknown`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
writetool failures all landed inerror_kind=unknownin BI. EveryWriteErrorsubclass (validation, backend, auth-denied, batch limits, unknown-op) was missing from_ERROR_KIND_TABLEinanalytics/wrappers.py.tool_args_invaliddidn't catch it:writes/dispatch.pycatchespydantic.ValidationErrorat everyop.pydantic_model.model_validate(...)call and re-raises asValidationFailedError, which extendsExceptiondirectly (notpydantic.ValidationError). So the pydantic row in the table never fired on the write path.WriteErrorsubclasses to the classifier (subclass-before-parent ordering preserved) plus aWriteErrorcatch-all row. Bucket names mirror the stableErrorCodeliterals already declared inwrites/errors.pyso BI dashboards can join the analytics event against the tool-result envelope by the same enum.This is a direct follow-up to PR #118 — discovered while verifying that 0.1.3 actually shrinks the 44% unknown share. Triggered 6 real failure paths through live MCP tool calls; 3 of them (every write-tool path) were misrouted to
unknown.Test plan
make checkclean — 508 tests pass, ruff format clean, mypy strict cleantest_analytics_wrappers.pycover everyWriteErrorsubclass + the catch-all rowtool_args_invalidrow preserved with updated comment explaining why it doesn't fire on the write pathwrite_validation_failed/write_backend_errorinstead ofunknown🤖 Generated with Claude Code