Add Coach-inspired session quality scoring and insights (#645)

This PR adds session-quality measurement to `agentsview`. It turns archived sessions into inspectable quality signals: deterministic scoring for behavior that can be measured locally, plus opt-in generated recommendations that summarize those facts without becoming canonical score data.

The core goal is explainable review. Quality labels should be backed by stored signal counts, filtered evidence, score explanations, and session drilldowns so reviewers can decide whether the measurements are useful before treating them as calibrated defaults.

## Source Inspiration

This work was shaped by Microsoft's [AI Engineering Coach](https://github.com/microsoft/AI-Engineering-Coach), especially its [rule catalog](https://github.com/microsoft/AI-Engineering-Coach/tree/main/src/core/rules), [scoring detector](https://github.com/microsoft/AI-Engineering-Coach/blob/main/src/core/detectors/scoring.ts), [insights analyzer](https://github.com/microsoft/AI-Engineering-Coach/blob/main/src/core/analyzer-insights.ts), and [metric definitions](https://github.com/microsoft/AI-Engineering-Coach/tree/main/src/core/metrics).

The `agentsview` implementation adapts those ideas to a local, multi-agent session archive. Coach-inspired rules such as lazy prompting, missing file context, repeated prompts, missing spec structure, runaway loops, prompt maturity, premium-model waste, and sustainable-work-pattern signals were reviewed and then sorted into deterministic scoring, aggregate analytics, generated-insight inputs, already-covered behavior, or blocked future work depending on the data available in `agentsview`.

That distinction is intentional: this PR borrows the framing and rule inventory, but it does not make Copilot/VS Code-specific assumptions or use an LLM as the source of truth for score computation.

## What This Adds

- A documented mapping from Coach rules to `agentsview` signal families, including what is scoreable now, what belongs in analytics or generated insights, and what is blocked by missing metadata.
- Deterministic prompt/context/workflow heuristics for local session scoring, including short task starts, missing criteria, missing verification, repeated prompts, missing code context, and repeated failing tool cycles.
- Persisted quality signal counts across SQLite, PostgreSQL, DuckDB, service, sync, and generated frontend API surfaces without recreating the archive database.
- Analytics and Insights surfaces for quality pattern summaries, grade/calibration metadata, signal totals, filtered evidence drilldowns, and deep links back to affected sessions.
- An opt-in canned LLM insight path using fixed templates, deterministic aggregate payloads, validation, provenance, cache keys, and template versions. Generated prose stays separate from health scores and signal rows.
- Rollout documentation, golden score assertions, resync preservation checks, frontend coverage, and screenshots so reviewers can evaluate the model before final calibration.

## Important Boundaries

- LLMs do **not** perform canonical scoring. Session health remains deterministic and locally computable.
- Generated recommendations are opt-in, cached, provenance-bearing, and kept out of score/signal writes.
- Coach rules that require missing metadata, such as agent mode, plan mode, slash commands, approval mode, AI-generated LOC, review timing, or request cancellation, are documented but not scored.
- Timing and work-pattern ideas such as weekend or late-night activity remain analytics material, not session-quality penalties.
- Archive migrations remain additive. Existing session, orphaned, trashed, and saved insight data must be preserved.

## Screenshots

![Quality patterns](https://github.com/kenn-io/agentsview/blob/quality/phase-6-validation-rollout/docs/screenshots/phase-6-quality-patterns.png?raw=true)

![Evidence drilldown](https://github.com/kenn-io/agentsview/blob/quality/phase-6-validation-rollout/docs/screenshots/phase-6-evidence-drilldown.png?raw=true)

![Generated insight archive](https://github.com/kenn-io/agentsview/blob/quality/phase-6-validation-rollout/docs/screenshots/phase-6-generated-insight.png?raw=true)

## Review Focus

- Whether the score-affecting heuristics are conservative enough and have acceptable false-positive behavior.
- Whether the scoring caps, grade deltas, and explanation fields are understandable enough for calibration.
- Whether the signal persistence/API contract is complete across SQLite, PostgreSQL, DuckDB, service, sync, and frontend clients.
- Whether the Insights page makes deterministic evidence easy to inspect before reading generated recommendation text.
- Whether generated insights are clearly bounded as optional summaries rather than a second source of scoring truth.
- Whether the rollout checklist is sufficient before enabling these semantics broadly on real archives.

## Remaining Rollout Work

Before treating this as final calibration, run a representative local archive pass and record score deltas, threshold crossings, false-positive themes, runtime impact, and source diversity. The added rollout document captures the expected report shape and the contract checklist for future signal/API changes.


Co-authored-by: Marius van Niekerk <mariusvniekerk@users.noreply.github.com>

Author: mariusvniekerk

docs/screenshots/phase-6-evidence-drilldown.png

@@ -0,0 +1,100 @@
+# Session Quality Validation And Rollout
+
+Phase 6 validates the deterministic scoring and Insights stack before broad
+rollout. Final calibration is intentionally gated on Phases 2-5 because the
+signal names, API fields, deterministic Insights views, and opt-in LLM insight
+flow must be stable before weights can be judged against real archives.
+
+## Rollout Gate
+
+Do not promote the new prompt, context, or workflow penalties as final default
+semantics until all of these are true:
+
+- Phase 2 scorer contracts are frozen, including signal names and penalty caps.
+- Phase 3 persistence/API fields are additive, nullable/backfilled, and covered
+  by resync preservation tests.
+- Phase 4 deterministic Insights views expose scored facts before generated
+  recommendation text.
+- Phase 5 LLM canned insights remain opt-in, cacheable, and separate from
+  canonical health score writes.
+- A local archive calibration pass has reviewed score deltas, threshold
+  crossings, and known false positives.
+
+## Validation Matrix
+
+| Area                      | Evidence                                                                                                                                   | Current scaffold                                                                        |
+| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- |
+| Score deltas              | Compare canonical score against a baseline with Coach-derived heuristics removed. Lock expected score, grade, basis, penalties, and delta. | `internal/signals/testdata/score_golden.json` and `TestComputeHealthScore_GoldenDeltas` |
+| Explanation fields        | Every score delta has inspectable `Basis` and `Penalties` entries.                                                                         | Golden score assertions                                                                 |
+| Archive preservation      | Full resync preserves source-missing/orphaned sessions and their computed quality signal columns.                                          | `TestResyncAllPreservesTrashedSessionData` orphan signal assertion                      |
+| Saved insights            | Full resync preserves saved insight rows.                                                                                                  | `TestResyncAllPreservesInsights`                                                        |
+| Deterministic Insights UI | Saved quality recommendations render provenance metadata and deterministic-score disclaimer.                                               | `frontend/e2e/insights-quality.spec.ts`                                                 |
+| LLM-disabled state        | Read-only or unavailable generation disables the Generate action and avoids background mutation.                                           | `frontend/e2e/insights-quality.spec.ts`                                                 |
+
+## Reference Repo Gap Checklist
+
+The `/tmp/AI-Engineering-Coach` reference flow has broader validation around
+analyzer contracts, page-level anti-pattern behavior, rule inventory, and reload
+stability. Before final rollout, add or explicitly waive these lanes:
+
+| Area                          | Why it matters                                                                                                                                                                                                                                                                                                               | Suggested evidence                                                                                                                                                |
+| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Aggregate quality contracts   | Coach validates analyzer outputs for empty data, prompt maturity, spec-driven behavior, intent classification, migration readiness, and recommendation structure. Agentsview has quality pattern transform tests, but rollout should pin the full `GetAnalyticsSignals` shape across empty, clean, and high-signal archives. | Backend SQLite and PG analytics-signals tests covering grade buckets, prompt/context/workflow/tool totals, trend buckets, and agent/project breakdowns.           |
+| False-positive corpus         | Score goldens lock expected deltas, but calibration needs explicit negative cases so terse expert prompts, planning-only discussions, pure Q&A, and code tasks with enough file/tool context are not penalized incorrectly.                                                                                                  | Golden fixtures with both positive and negative sessions per score-affecting signal.                                                                              |
+| Rule-to-signal inventory      | Coach has rule loader/compiler tests that catch catalog drift. Agentsview maps Coach rules into deterministic signals and analytics-only lanes, so the map should fail closed when a scored signal is undocumented or untested.                                                                                              | A lightweight docs/test check that every score-affecting `QualitySignals` field appears in the heuristic map, validation matrix, and at least one golden fixture. |
+| Deterministic dashboard e2e   | Coach e2e exercises anti-pattern score cards, pattern lists, context-management cards, empty states, and drill-downs. Current Phase 6 e2e mainly covers saved/generated insight behavior.                                                                                                                                    | Playwright coverage for the Quality Patterns section: summary cards, grade distribution, pattern cards, empty/error/loading states, and filter/date changes.      |
+| Calibration artifact          | The current template explains what to review, but there is no executable command that produces the review table.                                                                                                                                                                                                             | Local-only script or command that emits JSON/CSV with baseline score, new score, delta, grade crossing, top penalties, and high-delta session IDs.                |
+| Reload/performance guard      | Coach includes parser/analyzer memory and reload benchmarks. Quality scoring runs during resync/backfill and analytics queries, so large archives need a runtime guard before default-on rollout.                                                                                                                            | Documented benchmark command with archive size, resync/backfill duration, analytics-signals latency, and memory notes.                                            |
+| Parser/source diversity       | Reference tests cover multiple log sources before analyzer logic consumes them. Agentsview should ensure new quality signals are stable across the supported agents, not only Claude-shaped fixtures.                                                                                                                        | Signal fixtures from Codex, Claude, Cursor, Gemini, Kiro/OpenCode, and imported remote sessions where source formats differ.                                      |
+| LLM insight replay boundaries | Phase 5 keeps generated insights separate from canonical scoring, but rollout should validate cache/provenance edge cases.                                                                                                                                                                                                   | Tests for malformed or partial provenance, cache-hit vs fresh generation metadata, stale template versions, provider unavailable, and read-only archives.         |
+
+## Signal/API Contract Checklist
+
+Every analytics signal or persisted quality-signal change must update these
+surfaces together before review:
+
+- SQLite aggregation, signal evidence queries, and example extraction.
+- PostgreSQL aggregation/evidence parity, including stable timestamp formatting
+  for frontend deep links and evidence ordering.
+- `QualitySignals` versioning plus non-destructive stale-row recompute behavior
+  for archives that already completed the legacy backfill marker.
+- Server endpoint validation. Unknown `/analytics/signal-sessions` signals
+  should return `400` instead of silently producing empty evidence.
+- Frontend API types, pattern driver ids, labels, and evidence-panel state keyed
+  by the active analytics filters.
+- Contract tests for empty data, unsupported signals, filtered evidence, and at
+  least one positive example per score-affecting signal.
+- Documentation updates in this file and the heuristic map whenever semantics,
+  confidence, or score-affecting status changes.
+
+Outcome cohorts (`outcome_errored`, `outcome_abandoned`, `outcome_completed`)
+are valid drilldown groups, but they are not calibrated diagnostic signals
+because the calibration target already includes incomplete outcomes.
+
+## Calibration Report Template
+
+Run this after Phases 2-5 are finalized against a representative local archive.
+Keep the report local unless the data has been scrubbed.
+
+| Metric                       | Value | Notes                                                                            |
+| ---------------------------- | ----- | -------------------------------------------------------------------------------- |
+| Archive date range           | TBD   | Include machines/agents sampled.                                                 |
+| Sessions scored              | TBD   | Exclude unscored low-confidence unknown sessions.                                |
+| Median score delta           | TBD   | New score minus baseline without Coach-derived heuristics.                       |
+| p90 absolute delta           | TBD   | Review high-delta sessions manually.                                             |
+| Newly below A/B/C thresholds | TBD   | Count threshold crossings by grade.                                              |
+| Top penalty contributors     | TBD   | Rank by frequency and total points.                                              |
+| False-positive themes        | TBD   | Include examples such as terse expert prompts or planning-only code discussions. |
+| Runtime impact               | TBD   | Compare resync/backfill duration before and after signal computation.            |
+| Rollout decision             | TBD   | Default-on, preview-only, or shadow-only.                                        |
+
+## Operator Notes
+
+- New quality signal columns are additive; existing archives must not be deleted
+  or recreated to adopt them.
+- Full resync can recompute parser-derived signals and should preserve orphaned,
+  trashed, excluded, and saved insight data before swapping databases.
+- If score semantics shift materially, expose the new penalties through a
+  preview or shadow path first and document the delta in release notes.
+- LLM canned insights summarize deterministic aggregates only. They must not
+  write canonical score, grade, signal, or penalty rows.