Spec 038 — rule_fired trace event for Tier-3 rule fires (#251)

* Spec 038 — rule_fired trace event for Tier-3 rule fires

Closes the EC3-final watch-item: per-rule firing-rate audits are now a
1-line `jq '.kind=="rule_fired"'` over the trace file instead of a
multi-step content scan against `events.jsonl` agent tool outputs.

- `Suggestion.IsRule` (default false) discriminates Tier-3 from Tier-2.
- `TraceWriter.WriteRuleFired` emits `{ts, kind, rule, code, confidence,
  evidence, file, line, mode}` with the same 1024-char evidence
  truncation + path sanitization as diag rows.
- `CheckCommand.EmitDiagnostics` writes the event whenever a
  rule-flagged suggestion attaches; Tier-2 hits stay off the trace
  (their firing rate is visible via the opt-in MUR_TELEMETRY channel).
- 5 new unit tests: schema, evidence-truncation, path-sanitization,
  end-to-end pipeline emit-on-rule + no-emit-on-tier2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address Copilot CR feedback on #251

- Dispose JsonDocument instances explicitly in the new pipeline test;
  detach the rule_fired row via JsonElement.Clone() so assertions
  outlive the parsed doc without leaking pooled buffers.
- Fix the jq one-liner example: `'.kind=="rule_fired"'` evaluates the
  expression on every row (outputting a boolean), not filters to
  matching rows. Correct form is `'select(.kind=="rule_fired")'`.
  Fixed in both the spec/tasks watch-item entry and the TraceWriter
  test comment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: codemonkeychris

CHANGELOG.md

@@ -297,6 +297,15 @@ to land under these conventions; subsequent specs follow this shape.
   (Border / Image / ScrollView) with a back-reference to the Controls section
   — the prior parenthetical Button carve-out was easy to miss mid-build.
   (spec 038 §3 — agent-facing skill updates)
+- Spec 038 EC3-final watch-item: `rule_fired` trace event. When a Tier-3
+  rule attaches a suggestion to a diagnostic, `mur check --trace` now writes
+  one structured row per fire:
+  `{kind: "rule_fired", rule, code, confidence, evidence, file, line, mode}`.
+  Per-rule firing-rate audits collapse from multi-step content scans against
+  `events.jsonl` agent tool outputs to a 1-line `jq` over the trace file.
+  Tier-2 suggestions deliberately do not emit this row — Tier-2 firing rates
+  are visible via the opt-in `MUR_TELEMETRY=1` channel. (spec 038 §0.3,
+  EC3-final watch-item)
 - Spec 038 §3.1a residual: trace-channel structured warning hook for
   self-disabled rules. `TraceWriter.WriteRuleSelfDisabled(rule, target)`
   emits `{kind: "rule_self_disabled", rule, unresolved_target, mode}`.

docs/reference/mur-check-did-you-mean.md

@@ -375,7 +375,7 @@ CheckCommand.Run               (CheckCommand.cs)
  │                     ├─ for each rule: TargetsResolve (else self-disable callback)
  │                     └─ TryMatch; pick highest confidence
  │                 RULE WINS over tier2Best (spec §6)
- └─ trace.Write (optional, --trace; records all parsed diagnostics + rule self-disables)
+ └─ trace.Write (optional, --trace; records all parsed diagnostics + rule self-disables + rule fires)
 ```
 
 The `Suggestion` (highest-confidence above threshold, rule preferred over Tier-2) is attached to the `Diag` for line formatting in `Diag.Format`.
@@ -456,7 +456,11 @@ Phases 0–2 are merged to `main`. Phase 3 is in flight on `eval/spec-038-ec3-20
 
 ### Phase 0 — instrumentation (merged)
 
-- `--trace <path>` flag on `mur check`. Writes a JSONL stream of every parsed diagnostic, one row per diagnostic. Schema: `{ts, code, severity, file, line, col, msg, receiver_type?, member?, mode}`. Trace is opt-in, never written by default, never includes source code text, never includes absolute paths outside the project root.
+- `--trace <path>` flag on `mur check`. Writes a JSONL stream, one row per parsed diagnostic (plus auxiliary structured events). Trace is opt-in, never written by default, never includes source code text, never includes absolute paths outside the project root. Row kinds:
+  - **diagnostic row** (default, no `kind` field): `{ts, code, severity, file, line, col, msg, receiver_type?, member?, mode}`.
+  - **command header** (`kind: "command"`): `{ts, kind, argv, mode}` — full effective `dotnet build` argv at the head of the trace.
+  - **rule self-disabled** (`kind: "rule_self_disabled"`): `{ts, kind, rule, unresolved_target, mode}` — emitted when a Tier-3 rule's declared target fails to resolve against the live compilation. Dedup'd per-invocation per-rule.
+  - **rule fired** (`kind: "rule_fired"`): `{ts, kind, rule, code, confidence, evidence, file, line, mode}` — emitted whenever a Tier-3 rule attaches a suggestion to a diagnostic. Tier-2 hits do not emit this row; their firing rate is visible via the opt-in `MUR_TELEMETRY=1` channel.
 - Folder structure: `src/Reactor.Cli/Check/{Suggesters,Rules}/` with README pointers; mirrored test folders.
 - A smoke fixture (`tests/Reactor.IntegrationTests/MurCheck/Fixtures/SmokeFixture/`) plus a smoke test that drives the end-to-end pipeline.
 

docs/specs/tasks/038-mur-check-did-you-mean-implementation.md

@@ -545,7 +545,7 @@ Clean rerun after the EC3-original watch-items closed: template typo fix (`Micrs
 - **Class-A rule exercise.** Targeted-prompt batch at `C:\temp\mur-targeted-prompt-spec.md` is still the empirical question on the three new Class-A rules. Author or curate prompts that surface CS1955/`GridSize.Auto`, CS0117/`GridSize.Pixel|Pixels|Fixed`, and CS1061/CS0117 on `TextBlockElement.Style`. The clean-PASS doesn't change that this batch left the rules' token impact unmeasured.
 - **§11 risk-row guardrail retrofit.** Move EC3-final criterion #3 from "low-confidence audit" to "verified" by adding a post-run analysis pass that runs `mur check --final` against the run's final workspace and compares against iteration-mode suggestions. Same instrument the EC3-original results section called for; still open.
 - **Tier-2 SKILL.md trims.** Now empirically de-risked by EC3-final's PASS. The drag-and-drop relocation to `reactor-input` and the Context-pattern trim could push `reactor-getting-started` another ~65 lines lighter. Separate PR; no blockers.
-- **`rule_fired` trace event.** One-line addition to `TraceWriter.cs` + the orchestrator emission point would make per-rule firing-rate audits a one-line grep against the trace file instead of multi-step content scan against `events.jsonl` agent tool outputs. Cheap; lands before the targeted-prompt batch ideally.
+- **`rule_fired` trace event — LANDED 2026-05-12.** `TraceWriter.WriteRuleFired(rule, code, confidence, evidence, file, line)` emits `{kind: "rule_fired", rule, code, confidence, evidence, file, line, mode}`. `Suggestion.IsRule` (default false) discriminates Tier-3 hits from Tier-2; `CheckCommand.EmitDiagnostics` writes the trace event whenever a rule-flagged suggestion attaches. Three new unit tests in `TraceWriterTests.cs` (schema, evidence-truncation, path-sanitization) + two pipeline tests in `CheckCommandPipelineTests.cs` (writes-on-rule, no-write-on-tier2) lock the contract. Tier-2 hits stay off the trace by design — telemetry channel still owns Tier-2 rate; the trace event exists specifically to make Tier-3 firings discoverable without telemetry opt-in. Targeted-prompt batch is now a 1-line `jq 'select(.kind=="rule_fired")'` over the trace file.
 
 ---
 

src/Reactor.Cli/Check/CheckCommand.cs

@@ -260,7 +260,24 @@ internal static void EmitDiagnostics(IReadOnlyList<Diag> diagnostics, TextWriter
             if (stdoutFilter is not null && !stdoutFilter(d)) continue;
             var suggestion = suggest?.Invoke(d);
             stdout.WriteLine(d.Format(suggestion));
-            if (suggestion is not null) Telemetry.OnSuggestionEmitted(d.Code, suggestion);
+            if (suggestion is not null)
+            {
+                Telemetry.OnSuggestionEmitted(d.Code, suggestion);
+                // Spec 038 EC3-final watch-item: structured rule-fire events
+                // make per-rule firing-rate audits a 1-line grep over the
+                // trace file. Only emitted for Tier-3 rule hits; Tier-2 fires
+                // are visible via the opt-in telemetry channel instead.
+                if (suggestion.IsRule)
+                {
+                    trace?.WriteRuleFired(
+                        ruleName: suggestion.SuggesterName,
+                        code: d.Code,
+                        confidence: suggestion.Confidence,
+                        evidence: suggestion.Evidence,
+                        file: d.File,
+                        line: d.Line);
+                }
+            }
         }
     }
 

src/Reactor.Cli/Check/SuggesterOrchestrator.cs

@@ -22,7 +22,7 @@
 
 namespace Microsoft.UI.Reactor.Cli.Check;
 
-internal sealed record Suggestion(string Text, double Confidence, string Evidence, string SuggesterName);
+internal sealed record Suggestion(string Text, double Confidence, string Evidence, string SuggesterName, bool IsRule = false);
 
 internal sealed class SuggesterOrchestrator
 {
@@ -180,7 +180,7 @@ public SuggesterOrchestrator(
                 var evidence = string.IsNullOrEmpty(s.Evidence)
                     ? rule.Provenance
                     : $"{s.Evidence} ({rule.Provenance})";
-                return new Suggestion(s.Text!, s.Confidence, evidence, rule.Name);
+                return new Suggestion(s.Text!, s.Confidence, evidence, rule.Name, IsRule: true);
             }
         }
 

src/Reactor.Cli/Check/TraceWriter.cs

@@ -109,6 +109,44 @@ public void WriteRuleSelfDisabled(string ruleName, string unresolvedTarget)
         writer.WriteLine(json);
     }
 
+    /// <summary>
+    /// Spec 038 EC3-final watch-item: emit one structured row per Tier-3 rule
+    /// fire so per-rule firing-rate audits are a 1-line grep against the trace
+    /// JSONL instead of a multi-step content scan against agent tool outputs.
+    /// Schema:
+    ///
+    ///   { ts, kind: "rule_fired", rule, code, confidence, evidence, file, line, mode }
+    ///
+    /// `evidence` is truncated to <see cref="MaxMessageChars"/> to keep the row
+    /// under the 2 KB cap the writer enforces. Tier-2 suggestions deliberately
+    /// do not emit this row — Tier-2 firing rates are derivable from telemetry
+    /// (~/.mur/telemetry/*.jsonl); the trace event exists specifically to make
+    /// Tier-3 rule firings discoverable without telemetry opt-in.
+    /// </summary>
+    public void WriteRuleFired(
+        string ruleName,
+        string code,
+        double confidence,
+        string evidence,
+        string file,
+        int line)
+    {
+        var ev = evidence;
+        if (ev.Length > MaxMessageChars) ev = ev[..MaxMessageChars];
+        var row = new RuleFiredRow(
+            ts: DateTime.UtcNow.ToString("o"),
+            kind: "rule_fired",
+            rule: ruleName,
+            code: code,
+            confidence: confidence,
+            evidence: ev,
+            file: SanitizePath(file, projectRoot),
+            line: line,
+            mode: mode);
+        var json = JsonSerializer.Serialize(row, JsonOpts);
+        writer.WriteLine(json);
+    }
+
     static readonly JsonSerializerOptions JsonOpts = new()
     {
         DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingNull,
@@ -187,4 +225,15 @@ internal sealed record RuleSelfDisabledRow(
         string rule,
         string unresolved_target,
         string mode);
+
+    internal sealed record RuleFiredRow(
+        string ts,
+        string kind,
+        string rule,
+        string code,
+        double confidence,
+        string evidence,
+        string file,
+        int line,
+        string mode);
 }

tests/Reactor.Tests/CheckCommandTests/CheckCommandPipelineTests.cs

@@ -212,6 +212,99 @@ public void Ranker_filter_suppresses_stdout_but_trace_still_records_every_row()
         finally { try { File.Delete(tracePath); } catch { } }
     }
 
+    [Fact]
+    public void Emit_writes_rule_fired_trace_event_when_rule_suggestion_attached()
+    {
+        // Spec 038 EC3-final watch-item: rule fires must show up in the trace
+        // as a structured `rule_fired` row so per-rule firing-rate audits are
+        // a 1-line grep. End-to-end through EmitDiagnostics: when the suggest
+        // callback returns a rule-flagged Suggestion, the trace file gains a
+        // rule_fired row alongside the diagnostic row.
+        var diags = CheckCommand.ParseDiagnostics("""
+            X.cs(1,1): error CS1955: Non-invocable member [P.csproj]
+            """);
+        var stdout = new StringWriter();
+        var tracePath = Path.Combine(Path.GetTempPath(), "mur-check-rulefire-" + Guid.NewGuid() + ".jsonl");
+        try
+        {
+            using (var trace = TraceWriter.Open(tracePath, Path.GetFullPath(".")))
+            {
+                CheckCommand.EmitDiagnostics(diags, stdout, trace,
+                    suggest: _ => new Suggestion(
+                        Text: ".Auto (drop parens)",
+                        Confidence: 0.95,
+                        Evidence: "GridSize.Auto is a property (cluster:C0004)",
+                        SuggesterName: "GridSizeFactoryParensRule",
+                        IsRule: true));
+            }
+
+            var lines = File.ReadAllLines(tracePath);
+            Assert.Equal(2, lines.Length); // 1 diag row + 1 rule_fired row
+
+            // Find the rule_fired row and detach its element so the parsed
+            // JsonDocument can be disposed inside the loop (Copilot CR: avoid
+            // leaking pooled buffers across the assertion block).
+            JsonElement ruleFired = default;
+            var found = false;
+            foreach (var line in lines)
+            {
+                using var doc = JsonDocument.Parse(line);
+                if (doc.RootElement.TryGetProperty("kind", out var k) &&
+                    k.GetString() == "rule_fired")
+                {
+                    ruleFired = doc.RootElement.Clone();
+                    found = true;
+                    break;
+                }
+            }
+            Assert.True(found, "trace did not contain a rule_fired row.");
+            Assert.Equal("GridSizeFactoryParensRule", ruleFired.GetProperty("rule").GetString());
+            Assert.Equal("CS1955", ruleFired.GetProperty("code").GetString());
+            Assert.Equal(0.95, ruleFired.GetProperty("confidence").GetDouble());
+            Assert.Equal("X.cs", ruleFired.GetProperty("file").GetString());
+            Assert.Equal(1, ruleFired.GetProperty("line").GetInt32());
+
+            // The stdout line still carries the → try: suffix — trace event
+            // is in addition to, not in place of.
+            Assert.Contains(".Auto (drop parens)", stdout.ToString());
+        }
+        finally { try { File.Delete(tracePath); } catch { } }
+    }
+
+    [Fact]
+    public void Emit_does_not_write_rule_fired_for_tier2_suggestions()
+    {
+        // Tier-2 firing rates are visible via the opt-in telemetry channel;
+        // the trace's rule_fired row is Tier-3-only by design. A Tier-2 hit
+        // here must leave no rule_fired marker in the trace.
+        var diags = CheckCommand.ParseDiagnostics("""
+            X.cs(1,1): error CS1061: missing member [P.csproj]
+            """);
+        var stdout = new StringWriter();
+        var tracePath = Path.Combine(Path.GetTempPath(), "mur-check-tier2-" + Guid.NewGuid() + ".jsonl");
+        try
+        {
+            using (var trace = TraceWriter.Open(tracePath, Path.GetFullPath(".")))
+            {
+                CheckCommand.EmitDiagnostics(diags, stdout, trace,
+                    suggest: _ => new Suggestion(
+                        Text: "Button(label, onClick: x)",
+                        Confidence: 0.91,
+                        Evidence: "factory has Action onClick parameter",
+                        SuggesterName: "SymbolSuggester")); // IsRule default = false
+            }
+
+            var lines = File.ReadAllLines(tracePath);
+            // Exactly one row — the diagnostic — and it is NOT a rule_fired row.
+            Assert.Single(lines);
+            using var doc = JsonDocument.Parse(lines[0]);
+            var hasKind = doc.RootElement.TryGetProperty("kind", out var k);
+            if (hasKind)
+                Assert.NotEqual("rule_fired", k.GetString());
+        }
+        finally { try { File.Delete(tracePath); } catch { } }
+    }
+
     [Fact]
     public void Ranker_final_mode_emits_everything_to_stdout()
     {

tests/Reactor.Tests/CheckCommandTests/TraceWriterTests.cs

@@ -186,6 +186,77 @@ public void Rule_self_disabled_row_has_expected_schema()
         Assert.True(line.Length <= 2048, $"trace row was {line.Length} bytes — exceeds 2 KB cap.");
     }
 
+    [Fact]
+    public void Rule_fired_row_has_expected_schema()
+    {
+        // Spec 038 EC3-final watch-item: per-rule firing-rate audits should
+        // be a 1-line grep over the trace file. The row carries enough info
+        // to identify the rule, the diagnostic it fired on, the confidence,
+        // and the location — so a maintainer can `jq 'select(.kind=="rule_fired")'`
+        // and reconstruct firing rates without rejoining against the diag rows.
+        using var tmp = TempFile.Create();
+        using (var w = TraceWriter.Open(tmp.Path, Path.GetFullPath("."), mode: "iteration"))
+        {
+            w.WriteRuleFired(
+                ruleName: "GridSizeFactoryParensRule",
+                code: "CS1955",
+                confidence: 0.95,
+                evidence: "GridSize.Auto is a property, not a method (cluster:C0004)",
+                file: "Program.cs",
+                line: 42);
+        }
+
+        var line = File.ReadAllLines(tmp.Path).Single();
+        using var doc = JsonDocument.Parse(line);
+        Assert.Equal("rule_fired", doc.RootElement.GetProperty("kind").GetString());
+        Assert.Equal("GridSizeFactoryParensRule", doc.RootElement.GetProperty("rule").GetString());
+        Assert.Equal("CS1955", doc.RootElement.GetProperty("code").GetString());
+        Assert.Equal(0.95, doc.RootElement.GetProperty("confidence").GetDouble());
+        Assert.Contains("GridSize.Auto", doc.RootElement.GetProperty("evidence").GetString());
+        Assert.Equal("Program.cs", doc.RootElement.GetProperty("file").GetString());
+        Assert.Equal(42, doc.RootElement.GetProperty("line").GetInt32());
+        Assert.Equal("iteration", doc.RootElement.GetProperty("mode").GetString());
+        Assert.True(line.Length <= 2048, $"trace row was {line.Length} bytes — exceeds 2 KB cap.");
+    }
+
+    [Fact]
+    public void Rule_fired_evidence_is_truncated_under_2KB_cap()
+    {
+        // Same defensive truncation we apply to diag.msg: a runaway evidence
+        // string should never blow the 2 KB row budget.
+        var huge = new string('e', 8192);
+        using var tmp = TempFile.Create();
+        using (var w = TraceWriter.Open(tmp.Path, Path.GetFullPath(".")))
+            w.WriteRuleFired("R", "CS1061", 0.9, huge, "a.cs", 1);
+
+        var line = File.ReadAllLines(tmp.Path).Single();
+        Assert.True(line.Length <= 2048, $"trace row was {line.Length} bytes — exceeds 2 KB cap.");
+    }
+
+    [Fact]
+    public void Rule_fired_file_is_sanitized_like_diag_rows()
+    {
+        // Same path-leak protection that applies to diag rows: a rule fire
+        // outside the project root must not leak the absolute file path.
+        var root = Path.GetFullPath(Path.Combine(Path.GetTempPath(), "reactor-trace-rf-" + Guid.NewGuid()));
+        Directory.CreateDirectory(root);
+        try
+        {
+            var outside = Path.Combine(Path.GetTempPath(), "elsewhere-" + Guid.NewGuid(), "Bar.cs");
+            using var tmp = TempFile.Create();
+            using (var w = TraceWriter.Open(tmp.Path, root))
+                w.WriteRuleFired("R", "CS1061", 0.9, "ev", outside, 1);
+
+            var line = File.ReadAllLines(tmp.Path).Single();
+            using var doc = JsonDocument.Parse(line);
+            Assert.Equal("<external>", doc.RootElement.GetProperty("file").GetString());
+        }
+        finally
+        {
+            try { Directory.Delete(root, recursive: true); } catch { }
+        }
+    }
+
     sealed class TempFile : IDisposable
     {
         public string Path { get; }