Spec 038 — rule_fired trace event for Tier-3 rule fires#251
Merged
Conversation
Closes the EC3-final watch-item: per-rule firing-rate audits are now a
1-line `jq '.kind=="rule_fired"'` over the trace file instead of a
multi-step content scan against `events.jsonl` agent tool outputs.
- `Suggestion.IsRule` (default false) discriminates Tier-3 from Tier-2.
- `TraceWriter.WriteRuleFired` emits `{ts, kind, rule, code, confidence,
evidence, file, line, mode}` with the same 1024-char evidence
truncation + path sanitization as diag rows.
- `CheckCommand.EmitDiagnostics` writes the event whenever a
rule-flagged suggestion attaches; Tier-2 hits stay off the trace
(their firing rate is visible via the opt-in MUR_TELEMETRY channel).
- 5 new unit tests: schema, evidence-truncation, path-sanitization,
end-to-end pipeline emit-on-rule + no-emit-on-tier2.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
5 tasks
There was a problem hiding this comment.
Pull request overview
This PR adds a new rule_fired JSONL trace row to mur check --trace so Tier-3 rule firings are directly discoverable from trace output (without requiring telemetry opt-in), and updates tests/docs to lock in the schema and behavior.
Changes:
- Add
Suggestion.IsRule(defaultfalse) and set it only when a Tier-3 rule wins inSuggesterOrchestrator. - Add
TraceWriter.WriteRuleFired(...)and emitkind: "rule_fired"rows fromCheckCommand.EmitDiagnosticswhenIsRulesuggestions are attached. - Add unit + pipeline tests and update docs/CHANGELOG to describe the new trace row kind.
Show a summary per file
| File | Description |
|---|---|
| tests/Reactor.Tests/CheckCommandTests/TraceWriterTests.cs | Adds schema/truncation/path-sanitization unit tests for the new rule_fired trace row. |
| tests/Reactor.Tests/CheckCommandTests/CheckCommandPipelineTests.cs | Adds end-to-end tests verifying rule_fired emission for Tier-3 and non-emission for Tier-2. |
| src/Reactor.Cli/Check/TraceWriter.cs | Implements WriteRuleFired + RuleFiredRow with evidence truncation and path sanitization. |
| src/Reactor.Cli/Check/SuggesterOrchestrator.cs | Extends Suggestion with IsRule and marks only rule-winning suggestions as IsRule: true. |
| src/Reactor.Cli/Check/CheckCommand.cs | Emits rule_fired trace rows when an IsRule suggestion is attached to a diagnostic. |
| docs/specs/tasks/038-mur-check-did-you-mean-implementation.md | Marks the watch-item as landed and documents the implementation + tests. |
| docs/reference/mur-check-did-you-mean.md | Updates trace docs to enumerate rule_fired alongside other row kinds. |
| CHANGELOG.md | Adds a changelog entry describing the new rule_fired trace event. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 8/8 changed files
- Comments generated: 2
- Dispose JsonDocument instances explicitly in the new pipeline test; detach the rule_fired row via JsonElement.Clone() so assertions outlive the parsed doc without leaking pooled buffers. - Fix the jq one-liner example: `'.kind=="rule_fired"'` evaluates the expression on every row (outputting a boolean), not filters to matching rows. Correct form is `'select(.kind=="rule_fired")'`. Fixed in both the spec/tasks watch-item entry and the TraceWriter test comment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
Addressed both Copilot CR comments in 739403e:
27/27 trace + pipeline tests still pass. |
Collaborator
Author
|
CI failure on Integration Tests is unrelated flake —
Re-triggered the failed job via |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
{kind: \"rule_fired\", rule, code, confidence, evidence, file, line, mode}row tomur check --tracewhenever a Tier-3 rule attaches a suggestion to a diagnostic.events.jsonlagent tool outputs to a 1-linejq '.kind==\"rule_fired\"'over the trace file.MUR_TELEMETRY=1channel; the trace event exists specifically to make Tier-3 firings discoverable without telemetry opt-in.Changes
Code (3 files):
Suggestionrecord gainsbool IsRule = false(positional default; existing call sites unchanged).SuggesterOrchestratorsetsIsRule: trueonly at the rule-winning return site.TraceWriter.WriteRuleFired(...)with same 1024-char evidence truncation + path sanitization as diag rows.CheckCommand.EmitDiagnosticsinvokes it on rule-flagged suggestions.Tests (5 new):
TraceWriterTests: row-schema, evidence-truncation under 2 KB, file-path sanitization for<external>paths.CheckCommandPipelineTests: end-to-end emit-on-rule + no-emit-on-tier2.Docs:
Spec 038 — mur check did-you-mean.docs/reference/mur-check-did-you-mean.mdtrace-schema section expanded to enumerate all four row kinds (diagnostic, command, rule_self_disabled, rule_fired).Follow-up
Remaining EC3-final watch-items + Phase-4 cleanup work tracked in #252 (filed alongside this PR).
Test plan
dotnet build src/Reactor.Clisucceeds (.NET 10, warnings-as-errors clean).rule_self_disabledschema for consistency.🤖 Generated with Claude Code