feat(phase-1): resilience + parser tightening (95% high-confidence)

[RyanAlberts]

What

Closes #4. Resolves B008.

PR #4 lifts the high-confidence enrichment rate from 67% to 95% on the same W26 batch. Adds ycai resume (recover from interrupted runs) and ycai dashboard (re-render from existing artifacts at zero LLM cost). Captures raw model failures for audit so future B008-shaped problems surface fast.

Why

PR #3's full-batch run had a 23% schema-validation failure rate. That's too high to ship. The ycai resume and standalone ycai dashboard commands also unblock the v0.1 release plan — without resume, a quota wall mid-run would mean burning the partial work.

Root cause of the 23% (now 0%)

The raw failure logger surfaced it immediately: every "schema-validation-failure" was the model's rationale field exceeding our 400-char Field(max_length=400) cap. The model was being thorough; the schema was being unnecessarily strict on a non-load-bearing field. PR #4 truncates rather than rejects.

The lenient extension to ai_capability and tech_stack (drop unknown values, fall back to unclear if emptied) contributes a smaller win on top.

Strict enforcement preserved for the load-bearing fields: industry_primary, oss_posture, confidence, sources. The contract that numbers come from validated data hasn't changed.

Before/after on real W26 data

metric	PR #3	PR #4	delta
Total analyzed	124	124	–
High confidence	83 (67%)	118 (95%)	+35
Schema failures	29 (23%)	0 (0%)	-29
Genuine model lows	12	6	-6
Hallucinated source URLs	0	0	–

Top finding strengthens: 68 of 118 high-confidence W26 companies build agents (58%). "W26 is the agentic batch" is now defensible on a larger cohort with zero schema noise.

New CLI surfaces

• ycai resume <run-dir> — reads analyses.jsonl, identifies missing slugs from coverage.json, runs enrichment only on what's left. Crash-resilient because each completed analysis writes to JSONL immediately.
• ycai dashboard <run-dir> — re-renders the HTML from existing artifacts. Re-runs the cited-URL publish gate. Zero LLM cost. Useful for iterating on dashboard layout, or generating partial reports after a quota wall.
• Live progress during enrichment shows running high / medium / low counts so you can spot a regression mid-run.
• raw_failures.jsonl captures every failed model response (schema/cross-check/hallucinated-URL) for offline audit. Capped at 4000 chars per record.

Test plan

• 103 tests passing (12 new). Lenient parsing for both new fields, rationale/tagline truncation, raw-failure capture across multiple calls, no-op when no log path provided.
• Mypy --strict clean.
• make publish-check clean.
• W26 full-batch re-run captured under examples/output/dashboard-w26-pr4-2026-05-01.html and analyses-w26-pr4-2026-05-01.json.
• ycai resume smoke-tested on the W26 run dir (correctly detected all 124 slugs already enriched).
• ycai dashboard smoke-tested on the same run dir (re-rendered without LLM call).

Backlog status

• B008 resolved by this PR (rate dropped from 23% to 0%).
• B007 (depth=1 website crawl for tech_stack/oss_posture) remains open. The PR PR #4 — Typer CLI, Rich progress, hot-restart resume #4 numbers confirm it's still the next-biggest signal lever — unknown stack/OSS still dominates.

Acceptance

• make validate-p0 green
• make publish-check green
• LLM-path invariants preserved (load-bearing fields stay strict)
• Live smoke captured on real W26

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request enhances the resilience and accuracy of the enrichment pipeline by implementing lenient parsing, incremental writes, and new CLI commands for resuming runs and re-rendering dashboards. Key improvements include filtering unknown enum values and truncating long text fields to prevent schema validation failures. Feedback suggests addressing a potential file handle leak in the CLI and adding error handling to the raw failure logging to prevent non-critical I/O issues from crashing the process.

[gemini-code-assist]

The current implementation leaves a file handle open until garbage collection because raw_failure_path.open() is called without a context manager. For consistency with other parts of this file (e.g., lines 84 and 144), consider using read_text().splitlines() which handles closing the file automatically.

Suggested change

	failure_count = sum(1 for _ in raw_failure_path.open())
	failure_count = len(raw_failure_path.read_text().splitlines())

[gemini-code-assist]

To improve the resilience of the enrichment process, consider wrapping the file I/O in a try-except block. A failure in writing to the audit log (e.g., due to disk space or permission issues) should ideally not crash the entire analysis run, especially since this is a non-critical logging operation.

Suggested change

	path.parent.mkdir(parents=True, exist_ok=True)
	with path.open("a") as f:
	f.write(json.dumps(record) + "\n")
	try:
	path.parent.mkdir(parents=True, exist_ok=True)
	with path.open("a") as f:
	f.write(json.dumps(record) + "\n")
	except Exception as exc:
	log.warning("Failed to write raw failure log for %s: %s", slug, exc)