docs: update README and CHANGELOG for V2 release

Add What's New in V2 section highlighting bring-your-own-data,
interactive onboarding, knowledge system, self-learning, brand theming,
and 606 tests. Fix stale references to deleted scripts and setup docs.
Update counts (18 agents, 39 skills, 20 commands).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: shane530

CHANGELOG.md

@@ -2,6 +2,35 @@
 
 All notable changes to this project will be documented in this file.
 
+## [2.0.0] - 2026-02-23
+
+### Added
+- Interactive onboarding: `/setup` interview learns role, data sources, business context
+- Knowledge infrastructure: corrections, learnings, query archaeology, organization knowledge
+- Self-learning loop: feedback capture, correction logging, proven SQL pattern retrieval
+- YAML-based brand theming with WCAG-compliant palettes (`themes/brands/`)
+- Pipeline run tracking: `/runs` to list, inspect, compare, and clean up runs
+- Comms drafter agent for Slack/email/exec summary output
+- Business context system: glossary, metrics, products, teams per organization
+- Notion ingest skill for importing business context from Notion workspaces
+- Entity resolver for cross-dataset disambiguation
+- 8 new slash commands: `/setup`, `/runs`, `/business`, `/log-correction`, `/architect`, `/notion-ingest`, `/setup-dev-context`, `/compare-datasets`
+- 9 new skills: archaeology, feedback-capture, log-correction, setup, setup-dev-context, runs, business, notion-ingest, architect
+- 606 tests with synthetic fixtures (no external data dependencies)
+- Health check system for data connectivity diagnostics
+- Schema migration helpers for knowledge file versioning
+
+### Changed
+- Fully dataset-agnostic: agents resolve tables/columns from active manifest, not hardcoded names
+- Removed bundled NovaMart dataset — bring your own data with `/connect-data`
+- Removed legacy setup scripts (`download-data.sh`, `build-duckdb.sh`) and setup docs
+- Updated CLAUDE.md with V2 workflow, agent index, and skill table
+- Python requirement bumped to 3.10+
+
+### Fixed
+- Pipeline resume reliability improved with persistent state management
+- Chart palette now validates WCAG contrast ratios
+
 ## [1.0.0] - 2026-02-19
 
 ### Added

README.md

@@ -1,19 +1,33 @@
-# AI Analyst
+# AI Analyst v2
 
-> [!IMPORTANT]
-> **Note from Shane — February 22, 2026**
->
-> This repo was last pushed to on February 20. Since then, the system has grown significantly through testing on more complex, real-world datasets. Everything below works, but the version we're running locally is substantially ahead of what's here. Major update coming in the next couple of days.
-
-[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
 [![Claude Code Required](https://img.shields.io/badge/requires-Claude%20Code-blueviolet.svg)](https://claude.ai/code)
+[![Tests](https://img.shields.io/badge/tests-606%20passing-brightgreen.svg)](#)
 
 A complete AI analyst system powered by Claude Code. You ask a business question. Claude frames it, explores your data, finds the root cause, builds a story, and hands you a branded slide deck with speaker notes. The whole thing takes minutes, not days.
 
-Connect your own data with `/connect-data` or use the included example datasets.
+**Bring your own data.** No bundled datasets to configure — connect your CSVs, DuckDB, or warehouse with `/connect-data` and start analyzing immediately.
+
+**18** specialized agents | **39** auto-applied skills | **20** slash commands | DAG-based parallel execution | PDF + HTML export
+
+---
+
+## What's New in V2
 
-**17** specialized agents | **30** auto-applied skills | **14** slash commands | DAG-based parallel execution | PDF + HTML export
+V2 is a ground-up rebuild of the intelligence layer. The pipeline and agents from V1 still work the same way — you won't notice a difference in how you use it. What changed is everything underneath.
+
+| Area | V1 | V2 |
+|------|----|----|
+| **Data** | Bundled NovaMart e-commerce dataset | Bring your own — CSV, DuckDB, Postgres, BigQuery, Snowflake |
+| **Onboarding** | Manual setup, read the docs | `/setup` interview learns your role, data, and business context |
+| **Memory** | Stateless across sessions | Knowledge system persists corrections, learnings, query patterns, business glossary |
+| **Self-learning** | None | Captures feedback, logs corrections, retrieves proven SQL patterns — never repeats the same mistake |
+| **Theming** | Hardcoded chart style | YAML-based theme system with brand colors, WCAG-compliant palettes |
+| **Business context** | None | Organization knowledge base — glossary, metrics, products, teams. Notion ingest. |
+| **Pipeline** | Single run, restart on failure | Run tracking (`/runs`), reliable resume, comms drafter for Slack/email output |
+| **Testing** | Minimal | 606 tests with synthetic fixtures, no external data dependencies |
+| **Dataset coupling** | NovaMart table names hardcoded in agents | Fully dataset-agnostic — agents resolve from active manifest and schema |
 
 ---
 
@@ -46,8 +60,7 @@ npm install -g @anthropic-ai/claude-code
 ```bash
 git clone https://github.com/ai-analyst-lab/ai-analyst.git
 cd ai-analyst
-bash scripts/setup.sh
-bash scripts/download-data.sh
+pip install -e ".[dev]"
 ```
 
 **3. Start Claude Code**
@@ -56,19 +69,19 @@ bash scripts/download-data.sh
 claude
 ```
 
-**4. Try it**
+**4. Connect your data and go**
 
 ```
-Why is our conversion rate declining?
+/connect-data
 ```
 
-Or go straight to the full pipeline:
+Or skip the wizard and just ask a question with your data in a directory:
 
 ```
-/run-pipeline data_path=data/your_dataset/ question="Why is conversion dropping?"
+/run-pipeline data_path=data/my_csvs/ question="Why is conversion dropping?"
 ```
 
-For full setup (MotherDuck, MCP connections, troubleshooting): [setup/prerequisites.md](setup/prerequisites.md)
+For full setup details: [docs/setup-guide.md](docs/setup-guide.md)
 
 ---
 
@@ -88,7 +101,7 @@ Claude queries the data and returns an answer with a chart. Simple questions get
 /run-pipeline data_path=data/your_dataset/ question="What's driving the decline in conversion?"
 ```
 
-The pipeline runs 17 agents across 4 phases: Frame the question, Analyze the data, Build the story, Create the deck. You get a validated analysis, branded charts, a narrative, and a slide deck with speaker notes. Exports to PDF and HTML.
+The pipeline runs 18 agents across 4 phases: Frame the question, Analyze the data, Build the story, Create the deck. You get a validated analysis, branded charts, a narrative, and a slide deck with speaker notes. Exports to PDF and HTML.
 
 ### 3. Explore a dataset
 
@@ -118,7 +131,7 @@ Claude generates a chart following Storytelling with Data methodology: warm off-
 
 ## How It Works: The Pipeline
 
-When you run `/run-pipeline`, Claude orchestrates 17 agents across 4 phases:
+When you run `/run-pipeline`, Claude orchestrates 18 agents across 4 phases:
 
 ```
 1. FRAME              2. ANALYZE                          3. STORY                 4. DECK
@@ -144,7 +157,7 @@ You don't have to run the whole thing. Five execution plans let you run just the
 
 | Plan | Use When | What Runs |
 |------|----------|-----------|
-| `full_presentation` | Complete analysis to slide deck | All 17 agents |
+| `full_presentation` | Complete analysis to slide deck | All 18 agents |
 | `deep_dive` | Analysis without presentation | Phases 1-2 only |
 | `quick_chart` | Just need one chart | Chart Maker + Design Critic |
 | `refresh_deck` | Re-do the presentation layer | Phases 3-4 (reuses analysis) |
@@ -206,11 +219,19 @@ Tier 6 (sequential)           Storytelling --> Deck Creator --> Slide Review -->
 | `/datasets` | List all connected datasets | `/datasets` |
 | `/switch-dataset` | Change the active dataset | `/switch-dataset my_dataset` |
 | `/connect-data` | Add a new data source | `/connect-data` |
+| `/setup` | Interactive onboarding interview | `/setup` |
 | `/metrics` | Browse the metric dictionary | `/metrics conversion_rate` |
 | `/history` | View past analyses | `/history` |
 | `/patterns` | View recurring patterns | `/patterns --global` |
 | `/export` | Export results in various formats | `/export slides` or `/export email` or `/export slack` |
 | `/forecast` | Generate a time-series forecast | `/forecast` |
+| `/runs` | List, inspect, compare pipeline runs | `/runs` |
+| `/business` | Browse organization knowledge | `/business glossary` |
+| `/log-correction` | Log a data or methodology correction | `/log-correction` |
+| `/architect` | Multi-persona planning methodology | `/architect` |
+| `/notion-ingest` | Import business context from Notion | `/notion-ingest` |
+| `/compare-datasets` | Compare metrics across datasets | `/compare-datasets` |
+| `/setup-dev-context` | Add codebase context for dev teams | `/setup-dev-context` |
 
 Or just ask in plain English. "Show me conversion by device" works as well as any command.
 
@@ -241,32 +262,17 @@ Your Data --> chart_helpers.py --> Base Chart (150 DPI)
 - Collision detection checks for overlapping text with 3 auto-fix strategies: offset the label, reduce font size, or drop the least important label. Charts with unresolved collisions halt the pipeline.
 - The deck uses branded HTML components: KPI cards, finding cards, recommendation rows, so-what callouts, before/after panels, timelines, and more
 - A lint gate validates every deck before export: checks frontmatter completeness, HTML component usage (minimum 3 types), valid slide classes, slide count, and pacing
-
-**Helper modules:**
-
-| Module | What It Does |
-|--------|-------------|
-| `chart_helpers.py` | Core SWD functions: `swd_style()`, `highlight_bar()`, `highlight_line()`, `action_title()`, `annotate_point()`, `save_chart()`, `stacked_bar()`, `retention_heatmap()`, `funnel_waterfall()`, `check_label_collisions()` |
-| `stats_helpers.py` | Statistical tests: proportion tests, mean tests, Mann-Whitney, chi-squared, bootstrap CI, effect size interpretation |
-| `analytics_helpers.py` | Analytical utilities for segmentation, decomposition, and driver analysis |
-| `forecast_helpers.py` | Time-series forecasting with trend detection and seasonality |
-| `marp_linter.py` | Validates Marp decks: frontmatter, HTML components, slide classes, pacing rules |
-| `marp_export.py` | Exports decks to PDF and HTML via Marp CLI |
+- YAML-based theming with brand color overrides and WCAG-compliant palettes (see [docs/theming.md](docs/theming.md))
 
 ---
 
 ## Your Data
 
-### What ships with the repo
-
-| Dataset | Path | Description |
-|---------|------|-------------|
-| Hero | `data/hero/` | Guided exploration dataset |
-| Examples | `data/examples/` | Curated public datasets with README guides |
+This repo ships clean — no bundled datasets. Connect your own data and the system builds context around it.
 
 ### Connect your own
 
-Run `/connect-data` for a guided setup wizard. Supported sources:
+Run `/connect-data` for a guided setup wizard, or `/setup` for a full onboarding interview. Supported sources:
 
 - **CSV files** — drop them in a directory, point Claude at it
 - **DuckDB** — local or MotherDuck
@@ -276,6 +282,10 @@ Run `/connect-data` for a guided setup wizard. Supported sources:
 
 The system auto-profiles your data, creates schema documentation, notes data quirks, and remembers context across sessions in `.knowledge/datasets/`.
 
+### Example datasets
+
+Curated public datasets with README guides are available in `data/examples/`.
+
 ### Fallback chain
 
 If your primary connection fails, the system falls back automatically:
@@ -326,15 +336,15 @@ working/                                # Intermediate files (safe to delete)
 | Change how Claude thinks | Edit `CLAUDE.md` (the AI's persona, rules, workflow) |
 | Add a new skill | Create `.claude/skills/my-skill/skill.md`, reference it in `CLAUDE.md` |
 | Add a new agent | Create `agents/my-agent.md` using `agents/CONTRACT_TEMPLATE.md` as a starting point |
-| Change the slide theme | Edit `themes/analytics-light.css` or create a new theme CSS |
+| Change the slide theme | Create a YAML theme in `themes/brands/` (see [docs/theming.md](docs/theming.md)) |
 | Add deck components | Edit `templates/marp_components.md` (snippet library) |
 | Modify the pipeline | Edit `.claude/skills/run-pipeline/skill.md` (rules, checkpoints, execution) |
 | Add to the agent DAG | Edit `agents/registry.yaml` (dependencies, execution order) |
 
 ---
 
 <details>
-<summary><strong>All 17 Agents</strong> (click to expand)</summary>
+<summary><strong>All 18 Agents</strong> (click to expand)</summary>
 
 Agents are markdown prompt templates in the `agents/` directory. Each defines a multi-step workflow with `{{VARIABLES}}` that get filled in at runtime. To invoke one, ask Claude to run it or use `/run-pipeline` to orchestrate all of them.
 
@@ -378,6 +388,7 @@ Agents are markdown prompt templates in the `agents/` directory. Each defines a
 |-------|-------------|---------------|
 | storytelling | Converts findings into a stakeholder-ready narrative with executive summary, findings, insight, and recommendations | 15 |
 | deck-creator | Builds a branded Marp slide deck with HTML components, speaker notes, and correct theme styling | 16 |
+| comms-drafter | Generates stakeholder communications: Slack summary, email brief, exec summary | 19 |
 
 ### Standalone
 
@@ -390,7 +401,7 @@ Agents are markdown prompt templates in the `agents/` directory. Each defines a
 ---
 
 <details>
-<summary><strong>All 30 Skills</strong> (click to expand)</summary>
+<summary><strong>All 39 Skills</strong> (click to expand)</summary>
 
 Skills are instruction files in `.claude/skills/` that Claude follows automatically when a trigger condition matches. You don't invoke them manually. When you ask for a chart, the Visualization Patterns skill activates. When you start an analysis, the Data Quality Check skill runs.
 
@@ -404,6 +415,7 @@ These skills shape every interaction:
 | close-the-loop | Every recommendation gets a decision owner, success metric, follow-up date, and fallback plan |
 | data-quality-check | Validates data completeness and consistency before analysis begins |
 | data-profiling | Deep-profiles schema, distributions, temporal patterns, and anomalies |
+| feedback-capture | Captures user corrections and methodology guidance to the learnings system |
 | first-run-welcome | Adaptive onboarding for new users based on available data |
 | guardrails | Pairs every success metric with a guardrail metric; checks positive findings for trade-offs |
 | knowledge-bootstrap | Loads active dataset context, schema, quirks, and user profile at session start |
@@ -415,6 +427,7 @@ These skills shape every interaction:
 | tracking-gaps | Identifies when required data doesn't exist and produces instrumentation requests |
 | triangulation | Cross-references findings against multiple sources before presenting |
 | visualization-patterns | Ensures every chart follows SWD design standards |
+| archaeology | Retrieves proven SQL patterns from query archaeology before writing new queries |
 
 ### On-Demand (Slash Commands)
 
@@ -435,8 +448,15 @@ These activate when you use a command:
 | patterns | `/patterns` | View recurring patterns across analyses |
 | forecast | `/forecast` | Generate time-series forecasts |
 | compare-datasets | `/compare-datasets` | Compare metrics across two datasets |
+| setup | `/setup` | Interactive onboarding interview for profile, data, and business context |
+| setup-dev-context | `/setup-dev-context` | Add codebase context for dev teams |
+| runs | `/runs` | List, inspect, compare, and clean up pipeline runs |
+| business | `/business` | Browse organization knowledge (glossary, metrics, products, teams) |
+| log-correction | `/log-correction` | Deliberate correction logging for methodology fixes |
+| architect | `/architect` | Multi-persona planning methodology for new projects |
+| notion-ingest | `/notion-ingest` | Crawl Notion workspace to extract business context |
 
-### Presentation
+### Presentation & Knowledge
 
 | Skill | What It Does |
 |-------|-------------|
@@ -457,10 +477,12 @@ Python modules in `helpers/` that agents call during execution:
 | Module | What It Does |
 |--------|-------------|
 | `chart_helpers.py` | Core SWD charting: `swd_style()`, `highlight_bar()`, `highlight_line()`, `action_title()`, `annotate_point()`, `save_chart()`, `stacked_bar()`, `retention_heatmap()`, `sensitivity_table()`, `funnel_waterfall()`, `big_number_layout()`, `check_label_collisions()` |
+| `chart_palette.py` | WCAG-compliant color palettes with brand override support |
 | `chart_style_guide.md` | Full SWD reference: color palette, declutter checklist, chart decision tree, anti-patterns |
 | `analytics_chart_style.mplstyle` | Matplotlib style file: off-white background, no top/right spines, sans-serif, 150 DPI |
 | `marp_linter.py` | Validates Marp decks: frontmatter, HTML components, slide classes, pacing, title collisions |
 | `marp_export.py` | Exports Marp decks to PDF and HTML via Marp CLI with theme resolution |
+| `theme_loader.py` | YAML-based theme system with brand color loading and inheritance |
 
 ### Data and SQL
 
@@ -489,23 +511,39 @@ Python modules in `helpers/` that agents call during execution:
 | `structural_validator.py` | Layer 1: schema, primary keys, completeness checks |
 | `logical_validator.py` | Layer 2: aggregation consistency, trend logic |
 | `business_rules.py` | Layer 3: plausibility checks against domain rules |
+| `business_validation.py` | Business rule validation against organization knowledge |
 | `simpsons_paradox.py` | Layer 4: Simpson's Paradox scanner |
 | `confidence_scoring.py` | Synthesizes all 4 layers into an A-F confidence grade |
 
+### Knowledge & Context
+
+| Module | What It Does |
+|--------|-------------|
+| `context_loader.py` | Loads active dataset context, schema, quirks at session start |
+| `archaeology_helpers.py` | Query archaeology: retrieve and match proven SQL patterns |
+| `business_context.py` | Organization knowledge: glossary, metrics, products, teams |
+| `entity_resolver.py` | Disambiguates entity references across datasets |
+| `metric_validator.py` | Validates metric definitions against schema |
+| `schema_migration.py` | Handles schema version migrations for knowledge files |
+| `miss_rate_logger.py` | Tracks knowledge system miss rates for improvement |
+
 ### System
 
 | Module | What It Does |
 |--------|-------------|
-| `error_helpers.py` | Student-friendly error messages with suggestions |
+| `error_helpers.py` | Friendly error messages with suggestions |
+| `file_helpers.py` | Atomic file writes, content hashing, safe YAML I/O |
+| `health_check.py` | System health diagnostics for data connectivity and dependencies |
 | `lineage_tracker.py` | Tracks data lineage from source through transformations to findings |
+| `pipeline_state.py` | Pipeline state management for run tracking and resume |
 
 </details>
 
 ---
 
 ## Requirements
 
-- **Python 3.9+** (3.10+ recommended)
+- **Python 3.10+**
 - **Node.js 18+** (for Claude Code)
 - **Claude Code** with a [Claude Pro subscription](https://claude.ai/pro) ($20/month)
 - **Internet connection** (for Claude API and optional MotherDuck)
@@ -514,9 +552,8 @@ Python modules in `helpers/` that agents call during execution:
 
 ## Getting Help
 
-- **Setup issues:** [setup/prerequisites.md](setup/prerequisites.md)
-- **MCP configuration:** [setup/mcp-config.md](setup/mcp-config.md)
-- **MotherDuck setup:** [setup/motherduck-setup.md](setup/motherduck-setup.md)
+- **Setup guide:** [docs/setup-guide.md](docs/setup-guide.md)
+- **Theming:** [docs/theming.md](docs/theming.md)
 - **Questions or bugs:** Open a [GitHub Issue](https://github.com/ai-analyst-lab/ai-analyst/issues)
 
 ---