Merge pull request #965 from nteract/chatwithchart

Add AI interrogation and chart-suggestion features

Author: emeeks

.clinerules

@@ -267,6 +267,64 @@ Canvas scene builders read CSS variables via `getComputedStyle` on the canvas DO
 ## AI Features
 `onObservation`/`useChartObserver`, `toConfig`/`fromConfig`/`toURL`/`fromURL`/`copyConfig`/`configToJSX`, `validateProps(component, props)`, `diagnoseConfig(component, props)`, `exportChart(div, { format })`, `npx semiotic-ai --doctor`
 
+### Conversational Interrogation (`semiotic/ai`)
+Headless hook for "chat with the chart" interactions. The library ships no UI — bring your own chat surface.
+- **`useChartInterrogation({ data, onQuery, componentName?, props?, initialAnnotations? })`** → `{ ask(query), history, summary, annotations, loading, error, reset }`
+- **`onQuery: (query, context) => Promise<{ answer, annotations? }>`** — call your LLM here. `context` is `{ data, summary, componentName?, props? }`.
+- **`summary`**: LLM-friendly statistical summary (`rowCount`, per-field `{ min, max, mean, median }` for numerics, top-k for categoricals, ISO range for dates). Available before any ask().
+- **`annotations`**: Merged `initialAnnotations` + latest AI response. Wire to the chart's `annotations` prop for visual highlighting.
+- **`summarizeData(data, options?)`**: Standalone for server-side prompting or batch jobs.
+- **MCP Tool**: `interrogateChart(component, props, query)` returns the same statistical summary and AI-facing instructions.
+
+```jsx
+import { LineChart, useChartInterrogation } from "semiotic/ai"
+
+function InterrogatableChart({ data }) {
+  const { ask, history, annotations, loading } = useChartInterrogation({
+    data,
+    componentName: "LineChart",
+    props: { xAccessor: "month", yAccessor: "revenue" },
+    onQuery: async (query, { summary }) => {
+      const res = await myLLMCall(query, summary)
+      return { answer: res.text, annotations: res.highlights }
+    },
+  })
+  return (
+    <>
+      <LineChart data={data} xAccessor="month" yAccessor="revenue" annotations={annotations} />
+      <YourChatUI history={history} loading={loading} onAsk={ask} />
+    </>
+  )
+}
+```
+
+### Chart Capability Layer (`semiotic/ai`)
+Heuristic chart-suggestion engine. Charts ship capability descriptors next to their TSX files; the engine ranks them against a profiled dataset by intent. No LLM call required.
+
+- **`profileData(data, { rawInput?, seriesField? })`** → `ChartDataProfile` (extends `DataSummary`): candidate fields per role (x/y/series/category/size/time), distinct counts, monotonicity, structure detection (hierarchy/network/geo).
+- **`suggestCharts(data, { intent?, allow?, deny?, maxResults?, includeVariants?, minScore? })`** → ranked `Suggestion[]` with `{ component, family, importPath, variant?, score, intentScores, rubric, reasons, caveats, props }`. `props` is spreadable directly into the matching chart.
+- **`scoreChart(component, data, { intent?, variantKey? })`** → evaluate a specific chart for a dataset (does it fit, how well, why/why not).
+- **`useChartSuggestions(data, options)`** → memoized React hook returning `{ suggestions, profile }`.
+- **`registerChartCapability(capability)`** / **`unregisterChartCapability(name)`** — runtime registration for custom charts.
+- **Intent taxonomy**: 13 built-in intents (`trend`, `compare-series`, `compare-categories`, `rank`, `part-to-whole`, `distribution`, `correlation`, `flow`, `hierarchy`, `geo`, `outlier-detection`, `composition-over-time`, `change-detection`). Extend via `registerIntent(descriptor)`.
+- **Capability authoring**: create `Foo.capability.ts` next to `Foo.tsx`, then append to the registry in `src/components/ai/chartCapabilities.ts`. Each capability declares `family`, `rubric` (familiarity/accuracy/precision 1-5), `fits(profile)` gate, `intentScores`, optional `variants` with `intentDeltas`, and `buildProps(profile, variant)`.
+- **Variants encode that settings change what a chart is good for**: e.g. `StackedAreaChart`'s `streamgraph` variant boosts trend but penalizes part-to-whole.
+- **Interrogation tie-in**: pass `includeSuggestions: true` to `useChartInterrogation` and the same ranked list lands in `context.suggestions` for the LLM.
+- **MCP tool**: `suggestCharts(data, intent?)` returns the ranked list as structured content.
+
+```jsx
+import { useChartSuggestions, LineChart, BarChart, /* ... */ } from "semiotic/ai"
+
+const COMPONENT_MAP = { LineChart, BarChart, /* ... */ }
+function SuggestedChart({ data, intent }) {
+  const { suggestions } = useChartSuggestions(data, { intent })
+  const top = suggestions[0]
+  if (!top) return null
+  const Component = COMPONENT_MAP[top.component]
+  return <Component {...top.props} />
+}
+```
+
 
 ## AI Behavior Contracts
 

.cursorrules

@@ -267,6 +267,64 @@ Canvas scene builders read CSS variables via `getComputedStyle` on the canvas DO
 ## AI Features
 `onObservation`/`useChartObserver`, `toConfig`/`fromConfig`/`toURL`/`fromURL`/`copyConfig`/`configToJSX`, `validateProps(component, props)`, `diagnoseConfig(component, props)`, `exportChart(div, { format })`, `npx semiotic-ai --doctor`
 
+### Conversational Interrogation (`semiotic/ai`)
+Headless hook for "chat with the chart" interactions. The library ships no UI — bring your own chat surface.
+- **`useChartInterrogation({ data, onQuery, componentName?, props?, initialAnnotations? })`** → `{ ask(query), history, summary, annotations, loading, error, reset }`
+- **`onQuery: (query, context) => Promise<{ answer, annotations? }>`** — call your LLM here. `context` is `{ data, summary, componentName?, props? }`.
+- **`summary`**: LLM-friendly statistical summary (`rowCount`, per-field `{ min, max, mean, median }` for numerics, top-k for categoricals, ISO range for dates). Available before any ask().
+- **`annotations`**: Merged `initialAnnotations` + latest AI response. Wire to the chart's `annotations` prop for visual highlighting.
+- **`summarizeData(data, options?)`**: Standalone for server-side prompting or batch jobs.
+- **MCP Tool**: `interrogateChart(component, props, query)` returns the same statistical summary and AI-facing instructions.
+
+```jsx
+import { LineChart, useChartInterrogation } from "semiotic/ai"
+
+function InterrogatableChart({ data }) {
+  const { ask, history, annotations, loading } = useChartInterrogation({
+    data,
+    componentName: "LineChart",
+    props: { xAccessor: "month", yAccessor: "revenue" },
+    onQuery: async (query, { summary }) => {
+      const res = await myLLMCall(query, summary)
+      return { answer: res.text, annotations: res.highlights }
+    },
+  })
+  return (
+    <>
+      <LineChart data={data} xAccessor="month" yAccessor="revenue" annotations={annotations} />
+      <YourChatUI history={history} loading={loading} onAsk={ask} />
+    </>
+  )
+}
+```
+
+### Chart Capability Layer (`semiotic/ai`)
+Heuristic chart-suggestion engine. Charts ship capability descriptors next to their TSX files; the engine ranks them against a profiled dataset by intent. No LLM call required.
+
+- **`profileData(data, { rawInput?, seriesField? })`** → `ChartDataProfile` (extends `DataSummary`): candidate fields per role (x/y/series/category/size/time), distinct counts, monotonicity, structure detection (hierarchy/network/geo).
+- **`suggestCharts(data, { intent?, allow?, deny?, maxResults?, includeVariants?, minScore? })`** → ranked `Suggestion[]` with `{ component, family, importPath, variant?, score, intentScores, rubric, reasons, caveats, props }`. `props` is spreadable directly into the matching chart.
+- **`scoreChart(component, data, { intent?, variantKey? })`** → evaluate a specific chart for a dataset (does it fit, how well, why/why not).
+- **`useChartSuggestions(data, options)`** → memoized React hook returning `{ suggestions, profile }`.
+- **`registerChartCapability(capability)`** / **`unregisterChartCapability(name)`** — runtime registration for custom charts.
+- **Intent taxonomy**: 13 built-in intents (`trend`, `compare-series`, `compare-categories`, `rank`, `part-to-whole`, `distribution`, `correlation`, `flow`, `hierarchy`, `geo`, `outlier-detection`, `composition-over-time`, `change-detection`). Extend via `registerIntent(descriptor)`.
+- **Capability authoring**: create `Foo.capability.ts` next to `Foo.tsx`, then append to the registry in `src/components/ai/chartCapabilities.ts`. Each capability declares `family`, `rubric` (familiarity/accuracy/precision 1-5), `fits(profile)` gate, `intentScores`, optional `variants` with `intentDeltas`, and `buildProps(profile, variant)`.
+- **Variants encode that settings change what a chart is good for**: e.g. `StackedAreaChart`'s `streamgraph` variant boosts trend but penalizes part-to-whole.
+- **Interrogation tie-in**: pass `includeSuggestions: true` to `useChartInterrogation` and the same ranked list lands in `context.suggestions` for the LLM.
+- **MCP tool**: `suggestCharts(data, intent?)` returns the ranked list as structured content.
+
+```jsx
+import { useChartSuggestions, LineChart, BarChart, /* ... */ } from "semiotic/ai"
+
+const COMPONENT_MAP = { LineChart, BarChart, /* ... */ }
+function SuggestedChart({ data, intent }) {
+  const { suggestions } = useChartSuggestions(data, { intent })
+  const top = suggestions[0]
+  if (!top) return null
+  const Component = COMPONENT_MAP[top.component]
+  return <Component {...top.props} />
+}
+```
+
 
 ## AI Behavior Contracts
 

.github/copilot-instructions.md

@@ -267,6 +267,64 @@ Canvas scene builders read CSS variables via `getComputedStyle` on the canvas DO
 ## AI Features
 `onObservation`/`useChartObserver`, `toConfig`/`fromConfig`/`toURL`/`fromURL`/`copyConfig`/`configToJSX`, `validateProps(component, props)`, `diagnoseConfig(component, props)`, `exportChart(div, { format })`, `npx semiotic-ai --doctor`
 
+### Conversational Interrogation (`semiotic/ai`)
+Headless hook for "chat with the chart" interactions. The library ships no UI — bring your own chat surface.
+- **`useChartInterrogation({ data, onQuery, componentName?, props?, initialAnnotations? })`** → `{ ask(query), history, summary, annotations, loading, error, reset }`
+- **`onQuery: (query, context) => Promise<{ answer, annotations? }>`** — call your LLM here. `context` is `{ data, summary, componentName?, props? }`.
+- **`summary`**: LLM-friendly statistical summary (`rowCount`, per-field `{ min, max, mean, median }` for numerics, top-k for categoricals, ISO range for dates). Available before any ask().
+- **`annotations`**: Merged `initialAnnotations` + latest AI response. Wire to the chart's `annotations` prop for visual highlighting.
+- **`summarizeData(data, options?)`**: Standalone for server-side prompting or batch jobs.
+- **MCP Tool**: `interrogateChart(component, props, query)` returns the same statistical summary and AI-facing instructions.
+
+```jsx
+import { LineChart, useChartInterrogation } from "semiotic/ai"
+
+function InterrogatableChart({ data }) {
+  const { ask, history, annotations, loading } = useChartInterrogation({
+    data,
+    componentName: "LineChart",
+    props: { xAccessor: "month", yAccessor: "revenue" },
+    onQuery: async (query, { summary }) => {
+      const res = await myLLMCall(query, summary)
+      return { answer: res.text, annotations: res.highlights }
+    },
+  })
+  return (
+    <>
+      <LineChart data={data} xAccessor="month" yAccessor="revenue" annotations={annotations} />
+      <YourChatUI history={history} loading={loading} onAsk={ask} />
+    </>
+  )
+}
+```
+
+### Chart Capability Layer (`semiotic/ai`)
+Heuristic chart-suggestion engine. Charts ship capability descriptors next to their TSX files; the engine ranks them against a profiled dataset by intent. No LLM call required.
+
+- **`profileData(data, { rawInput?, seriesField? })`** → `ChartDataProfile` (extends `DataSummary`): candidate fields per role (x/y/series/category/size/time), distinct counts, monotonicity, structure detection (hierarchy/network/geo).
+- **`suggestCharts(data, { intent?, allow?, deny?, maxResults?, includeVariants?, minScore? })`** → ranked `Suggestion[]` with `{ component, family, importPath, variant?, score, intentScores, rubric, reasons, caveats, props }`. `props` is spreadable directly into the matching chart.
+- **`scoreChart(component, data, { intent?, variantKey? })`** → evaluate a specific chart for a dataset (does it fit, how well, why/why not).
+- **`useChartSuggestions(data, options)`** → memoized React hook returning `{ suggestions, profile }`.
+- **`registerChartCapability(capability)`** / **`unregisterChartCapability(name)`** — runtime registration for custom charts.
+- **Intent taxonomy**: 13 built-in intents (`trend`, `compare-series`, `compare-categories`, `rank`, `part-to-whole`, `distribution`, `correlation`, `flow`, `hierarchy`, `geo`, `outlier-detection`, `composition-over-time`, `change-detection`). Extend via `registerIntent(descriptor)`.
+- **Capability authoring**: create `Foo.capability.ts` next to `Foo.tsx`, then append to the registry in `src/components/ai/chartCapabilities.ts`. Each capability declares `family`, `rubric` (familiarity/accuracy/precision 1-5), `fits(profile)` gate, `intentScores`, optional `variants` with `intentDeltas`, and `buildProps(profile, variant)`.
+- **Variants encode that settings change what a chart is good for**: e.g. `StackedAreaChart`'s `streamgraph` variant boosts trend but penalizes part-to-whole.
+- **Interrogation tie-in**: pass `includeSuggestions: true` to `useChartInterrogation` and the same ranked list lands in `context.suggestions` for the LLM.
+- **MCP tool**: `suggestCharts(data, intent?)` returns the ranked list as structured content.
+
+```jsx
+import { useChartSuggestions, LineChart, BarChart, /* ... */ } from "semiotic/ai"
+
+const COMPONENT_MAP = { LineChart, BarChart, /* ... */ }
+function SuggestedChart({ data, intent }) {
+  const { suggestions } = useChartSuggestions(data, { intent })
+  const top = suggestions[0]
+  if (!top) return null
+  const Component = COMPONENT_MAP[top.component]
+  return <Component {...top.props} />
+}
+```
+
 
 ## AI Behavior Contracts
 

.windsurfrules

@@ -267,6 +267,64 @@ Canvas scene builders read CSS variables via `getComputedStyle` on the canvas DO
 ## AI Features
 `onObservation`/`useChartObserver`, `toConfig`/`fromConfig`/`toURL`/`fromURL`/`copyConfig`/`configToJSX`, `validateProps(component, props)`, `diagnoseConfig(component, props)`, `exportChart(div, { format })`, `npx semiotic-ai --doctor`
 
+### Conversational Interrogation (`semiotic/ai`)
+Headless hook for "chat with the chart" interactions. The library ships no UI — bring your own chat surface.
+- **`useChartInterrogation({ data, onQuery, componentName?, props?, initialAnnotations? })`** → `{ ask(query), history, summary, annotations, loading, error, reset }`
+- **`onQuery: (query, context) => Promise<{ answer, annotations? }>`** — call your LLM here. `context` is `{ data, summary, componentName?, props? }`.
+- **`summary`**: LLM-friendly statistical summary (`rowCount`, per-field `{ min, max, mean, median }` for numerics, top-k for categoricals, ISO range for dates). Available before any ask().
+- **`annotations`**: Merged `initialAnnotations` + latest AI response. Wire to the chart's `annotations` prop for visual highlighting.
+- **`summarizeData(data, options?)`**: Standalone for server-side prompting or batch jobs.
+- **MCP Tool**: `interrogateChart(component, props, query)` returns the same statistical summary and AI-facing instructions.
+
+```jsx
+import { LineChart, useChartInterrogation } from "semiotic/ai"
+
+function InterrogatableChart({ data }) {
+  const { ask, history, annotations, loading } = useChartInterrogation({
+    data,
+    componentName: "LineChart",
+    props: { xAccessor: "month", yAccessor: "revenue" },
+    onQuery: async (query, { summary }) => {
+      const res = await myLLMCall(query, summary)
+      return { answer: res.text, annotations: res.highlights }
+    },
+  })
+  return (
+    <>
+      <LineChart data={data} xAccessor="month" yAccessor="revenue" annotations={annotations} />
+      <YourChatUI history={history} loading={loading} onAsk={ask} />
+    </>
+  )
+}
+```
+
+### Chart Capability Layer (`semiotic/ai`)
+Heuristic chart-suggestion engine. Charts ship capability descriptors next to their TSX files; the engine ranks them against a profiled dataset by intent. No LLM call required.
+
+- **`profileData(data, { rawInput?, seriesField? })`** → `ChartDataProfile` (extends `DataSummary`): candidate fields per role (x/y/series/category/size/time), distinct counts, monotonicity, structure detection (hierarchy/network/geo).
+- **`suggestCharts(data, { intent?, allow?, deny?, maxResults?, includeVariants?, minScore? })`** → ranked `Suggestion[]` with `{ component, family, importPath, variant?, score, intentScores, rubric, reasons, caveats, props }`. `props` is spreadable directly into the matching chart.
+- **`scoreChart(component, data, { intent?, variantKey? })`** → evaluate a specific chart for a dataset (does it fit, how well, why/why not).
+- **`useChartSuggestions(data, options)`** → memoized React hook returning `{ suggestions, profile }`.
+- **`registerChartCapability(capability)`** / **`unregisterChartCapability(name)`** — runtime registration for custom charts.
+- **Intent taxonomy**: 13 built-in intents (`trend`, `compare-series`, `compare-categories`, `rank`, `part-to-whole`, `distribution`, `correlation`, `flow`, `hierarchy`, `geo`, `outlier-detection`, `composition-over-time`, `change-detection`). Extend via `registerIntent(descriptor)`.
+- **Capability authoring**: create `Foo.capability.ts` next to `Foo.tsx`, then append to the registry in `src/components/ai/chartCapabilities.ts`. Each capability declares `family`, `rubric` (familiarity/accuracy/precision 1-5), `fits(profile)` gate, `intentScores`, optional `variants` with `intentDeltas`, and `buildProps(profile, variant)`.
+- **Variants encode that settings change what a chart is good for**: e.g. `StackedAreaChart`'s `streamgraph` variant boosts trend but penalizes part-to-whole.
+- **Interrogation tie-in**: pass `includeSuggestions: true` to `useChartInterrogation` and the same ranked list lands in `context.suggestions` for the LLM.
+- **MCP tool**: `suggestCharts(data, intent?)` returns the ranked list as structured content.
+
+```jsx
+import { useChartSuggestions, LineChart, BarChart, /* ... */ } from "semiotic/ai"
+
+const COMPONENT_MAP = { LineChart, BarChart, /* ... */ }
+function SuggestedChart({ data, intent }) {
+  const { suggestions } = useChartSuggestions(data, { intent })
+  const top = suggestions[0]
+  if (!top) return null
+  const Component = COMPONENT_MAP[top.component]
+  return <Component {...top.props} />
+}
+```
+
 
 ## AI Behavior Contracts