Add AI interrogation and chart-suggestion features#965
Merged
Conversation
Introduce a headless conversational interrogation API and a chart capability/suggestion layer under semiotic/ai. Adds useChartInterrogation, data summarization, profileData, suggestCharts, scoreChart, useChartSuggestions, registration APIs, MCP tooling (interrogateChart/suggestCharts) and server integration. Includes many new TypeScript implementations and tests, a large set of chart capability descriptors, docs and examples (blog entries, pages, and README/guide updates), new scorecard scripts and a capability-scorecard JSON, an IMPROVEMENTS roadmap, and minor metadata updates (package.json, changelog entry for 3.6.0, server.json). Enables LLM-driven "chat with the chart" workflows and heuristic chart recommendations.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands Semiotic’s AI surface (semiotic/ai) with a headless “chat with the chart” interrogation hook, a heuristic chart capability/suggestion engine (including audience-aware and streaming-schema variants), plus supporting tests, tooling, and docs/navigation updates for the new “Intelligence” section.
Changes:
- Added conversational interrogation utilities (
useChartInterrogation,useChartFocus,summarizeData) and a data summarizer for LLM-friendly context. - Introduced a chart capability registry with per-chart descriptors, intent taxonomy, audience bias + stretch suggestions, streaming-schema suggestions, and config repair/scorecard helpers (with tests and CI scripts).
- Updated docs site routing/nav/blog tooling and bumped version metadata to
3.6.0.
Reviewed changes
Copilot reviewed 116 out of 120 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/components/store/useChartFocus.ts | New hook to derive interrogation focus from observation events. |
| src/components/store/useChartFocus.test.tsx | Tests for focus derivation and filtering behavior. |
| src/components/semiotic-ai.ts | Public semiotic/ai export surface for new AI features and capability APIs. |
| src/components/data/DataSummarizer.ts | New dataset summarizer for LLM prompting (stats + samples). |
| src/components/data/DataSummarizer.test.ts | Tests for summarization across numeric/categorical/date/edge cases. |
| src/components/charts/xy/StackedAreaChart.capability.ts | Capability descriptor + variants for stacked area charts. |
| src/components/charts/xy/Scatterplot.capability.ts | Capability descriptor + variants for scatterplots. |
| src/components/charts/xy/QuadrantChart.capability.ts | Capability descriptor and default quadrant centers. |
| src/components/charts/xy/MultiAxisLineChart.capability.ts | Capability descriptor for dual-axis line charts. |
| src/components/charts/xy/MinimapChart.capability.ts | Capability descriptor for minimap charts. |
| src/components/charts/xy/LineChart.capability.ts | Capability descriptor + variants for line charts. |
| src/components/charts/xy/Heatmap.capability.ts | Capability descriptor + variants and stricter fit gating for heatmaps. |
| src/components/charts/xy/DifferenceChart.capability.ts | Capability descriptor for difference charts. |
| src/components/charts/xy/ConnectedScatterplot.capability.ts | Capability descriptor for connected scatterplots. |
| src/components/charts/xy/CandlestickChart.capability.ts | Capability descriptor for candlestick charts. |
| src/components/charts/xy/BubbleChart.capability.ts | Capability descriptor for bubble charts. |
| src/components/charts/xy/AreaChart.capability.ts | Capability descriptor + variants for area charts. |
| src/components/charts/realtime/TemporalHistogram.capability.ts | Streaming capability descriptor for TemporalHistogram. |
| src/components/charts/realtime/RealtimeWaterfallChart.capability.ts | Streaming capability descriptor for RealtimeWaterfallChart. |
| src/components/charts/realtime/RealtimeSwarmChart.capability.ts | Streaming capability descriptor for RealtimeSwarmChart. |
| src/components/charts/realtime/RealtimeLineChart.capability.ts | Streaming capability descriptor for RealtimeLineChart. |
| src/components/charts/realtime/RealtimeHistogram.capability.ts | Streaming capability descriptor for RealtimeHistogram. |
| src/components/charts/realtime/RealtimeHeatmap.capability.ts | Streaming capability descriptor for RealtimeHeatmap. |
| src/components/charts/ordinal/ViolinPlot.capability.ts | Capability descriptor + variants for violin plots. |
| src/components/charts/ordinal/SwimlaneChart.capability.ts | Capability descriptor for swimlane charts. |
| src/components/charts/ordinal/SwarmPlot.capability.ts | Capability descriptor for swarm plots. |
| src/components/charts/ordinal/StackedBarChart.capability.ts | Capability descriptor + variants for stacked bars. |
| src/components/charts/ordinal/RidgelinePlot.capability.ts | Capability descriptor for ridgeline plots. |
| src/components/charts/ordinal/PieChart.capability.ts | Capability descriptor + variants for pie charts. |
| src/components/charts/ordinal/LikertChart.capability.ts | Capability descriptor with rating-field heuristics. |
| src/components/charts/ordinal/Histogram.tsx | Adjusted default category accessor to support raw observation histograms. |
| src/components/charts/ordinal/Histogram.test.tsx | Regression test for raw-observation histogram rendering/validation. |
| src/components/charts/ordinal/Histogram.capability.ts | Capability descriptor + variants for histograms. |
| src/components/charts/ordinal/GroupedBarChart.capability.ts | Capability descriptor for grouped bar charts. |
| src/components/charts/ordinal/GaugeChart.capability.ts | Capability descriptor for gauge charts. |
| src/components/charts/ordinal/FunnelChart.capability.ts | Capability descriptor with stage-field heuristics. |
| src/components/charts/ordinal/DotPlot.capability.ts | Capability descriptor for dot plots. |
| src/components/charts/ordinal/DonutChart.capability.ts | Capability descriptor for donut charts. |
| src/components/charts/ordinal/BoxPlot.capability.ts | Capability descriptor for box plots. |
| src/components/charts/ordinal/BarChart.capability.ts | Capability descriptor + variants for bar charts. |
| src/components/charts/network/Treemap.capability.ts | Capability descriptor for treemaps. |
| src/components/charts/network/TreeDiagram.capability.ts | Capability descriptor + variants for tree diagrams. |
| src/components/charts/network/SankeyDiagram.capability.ts | Capability descriptor for sankey diagrams. |
| src/components/charts/network/ProcessSankey.capability.ts | Capability descriptor for temporal/process sankey. |
| src/components/charts/network/OrbitDiagram.capability.ts | Capability descriptor for orbit diagrams. |
| src/components/charts/network/ForceDirectedGraph.capability.ts | Capability descriptor for force-directed graphs. |
| src/components/charts/network/CirclePack.capability.ts | Capability descriptor for circle pack charts. |
| src/components/charts/network/ChordDiagram.capability.ts | Capability descriptor for chord diagrams. |
| src/components/charts/geo/ProportionalSymbolMap.capability.ts | Capability descriptor for proportional symbol maps. |
| src/components/charts/geo/FlowMap.capability.ts | Capability descriptor for flow maps. |
| src/components/charts/geo/DistanceCartogram.capability.ts | Capability descriptor for distance cartograms. |
| src/components/charts/geo/ChoroplethMap.capability.ts | Capability descriptor for choropleth maps. |
| src/components/ai/useChartSuggestions.ts | React hook wrapper around profiling + suggestion generation. |
| src/components/ai/suggestStretchCharts.ts | Audience-based “stretch” suggestion surface. |
| src/components/ai/suggestStretchCharts.test.ts | Tests for stretch suggestions and dashboard stretch panels. |
| src/components/ai/suggestStreamCharts.ts | Streaming-schema chart recommendation engine + registry APIs. |
| src/components/ai/suggestStreamCharts.test.ts | Tests for streaming suggestions and registration override behavior. |
| src/components/ai/suggestDashboard.test.ts | Tests for dashboard panel selection/diversification behavior. |
| src/components/ai/streamingTypes.ts | Types for stream schema, stream capabilities, and suggestions. |
| src/components/ai/repairChartConfig.ts | Chart “repair” API to validate a choice and suggest alternatives. |
| src/components/ai/repairChartConfig.test.ts | Tests for repair outcomes (ok/alternative/unknown). |
| src/components/ai/qualityScorecard.test.ts | Tests for quality scorecard report shape and agreement thresholds. |
| src/components/ai/profileData.test.ts | Tests for data profiling (time/series/categorical/rawInput detection). |
| src/components/ai/intents.ts | Intent taxonomy + runtime registration APIs. |
| src/components/ai/inferIntent.test.ts | Tests for query→intent inference behavior. |
| src/components/ai/diffProfile.ts | Profile diff utility to explain suitability changes across refreshes. |
| src/components/ai/diffProfile.test.ts | Tests for profile diff outputs and fit/unfit transitions. |
| src/components/ai/chartCapabilities.ts | Registry of built-in capability descriptors + registration APIs. |
| src/components/ai/audiences.ts | Built-in example audience personas (exec/analyst/data scientist). |
| src/components/ai/audienceProfile.ts | Audience bias model + familiarity/stretch helpers. |
| src/components/ai/audienceProfile.test.ts | Tests for bias math and audience-aware ranking effects. |
| server.json | Bumped MCP/server metadata version to 3.6.0. |
| scripts/scorecard-dev.ts | Dev script to run capability scorecard against TS source. |
| scripts/run-capability-scorecard.mjs | Node script to run scorecard on dist bundle and write JSON report. |
| scripts/check-capability-coverage.mjs | CI script to enforce capability descriptor coverage vs inventory. |
| scripts/check-blog-entry-sync.mjs | Enhanced blog sync check to read draft flags and source-of-truth arrays. |
| package.json | Version bump + new scorecard/coverage scripts + expanded subpath aliases. |
| package-lock.json | Lockfile version bump alignment. |
| IMPROVEMENTS.md | New roadmap/strategy document. |
| docs/src/pages/features/VegaLiteTranslatorPage.js | Docs navigation/breadcrumb updates to new “Intelligence” section. |
| docs/src/pages/features/SerializationPage.js | Docs navigation/breadcrumb updates to new “Intelligence” section. |
| docs/src/pages/features/PushApiPage.js | Docs navigation link update to new intelligence flow. |
| docs/src/pages/features/ObservationHooksPage.js | Docs page renaming + moved under “Intelligence” navigation. |
| docs/src/pages/features/CapabilitiesPage.js | Docs navigation/breadcrumb updates to new “Intelligence” section. |
| docs/src/components/navData.js | Added “Intelligence” nav group and moved AI-related pages into it. |
| docs/src/blog/entries.js | Added draft-aware entry registries and draft inclusion for direct routing. |
| docs/src/blog/entries-meta.js | Draft-aware meta registry + published-only filtered export. |
| docs/src/blog/components/BlogEntryView.js | Draft banner UI for unlisted entries. |
| docs/src/App.js | Added /intelligence routes and redirects from legacy /features paths. |
| docs/public/llms-full.txt | Updated LLM-facing docs content for interrogation + capability layer APIs. |
| docs/public/blog/feed.xml | Updated blog feed entries/metadata for latest post and author fields. |
| CLAUDE.md | Updated AI feature documentation for new interrogation/suggestion APIs. |
| CHANGELOG.md | Added 3.6.0 heading for upcoming release. |
| ai/schema.json | Bumped AI schema version to 3.6.0. |
| .windsurfrules | Updated AI feature documentation block. |
| .github/copilot-instructions.md | Updated AI feature documentation block. |
| .cursorrules | Updated AI feature documentation block. |
| .clinerules | Updated AI feature documentation block. |
Multiple changes across docs, scripts, AI components, fixtures and tests:
- docs: add eslint-disable react/no-unescaped-entities to several blog entries and fix small text wrapping.
- scripts/check-capability-coverage.mjs: reformat path/regex strings, improve diagnostic note formatting and expand console summary to show covered charts count.
- tests: update MCP tools/list expectation to match registered tools and adjust audienceProfile tests imports/formatting.
- ai: simplify suggestStretchCharts logic to use a single top familiar pick as anchor; remove unused top-by-component map.
- capabilities: remove unused OHLC constant from Candlestick capability.
- DifferenceChart.capability: pivot long-form two-series data into wide {x,a,b} rows in buildProps and expose series labels so the chart receives the expected shape.
- DataSummarizer: detect numeric strings (e.g. "42", "3.14e6") as numeric so summaries preserve numeric stats.
- useChartFocus: normalize interaction payloads (selection/hover/click), handle additional end types (click-end), and ensure consistent focus datum shape.
- qualityFixtures: reformat and normalize fixture data objects/arrays for readability (no functional changes intended).
Overall these edits clean up code, improve robustness when handling common input shapes, and align tests with current capabilities.
Comment on lines
+174
to
+180
| // Default is a function rather than the string "category" so raw-observation | ||
| // mode works on data like [{ value: 12 }, { value: 18 }] (where no | ||
| // `category` field exists). Reads the row's `category` field when present | ||
| // and falls back to a single "All" bucket otherwise — so both pre-binned | ||
| // data with categories and raw observations render without configuration. | ||
| categoryAccessor = ((d: Datum) => (d?.category as string | undefined) ?? "All") as ChartAccessor<TDatum, string>, | ||
| valueAccessor = "value", |
Comment on lines
+10
to
+16
| /** | ||
| * Which observation types count as "focused." Default is hover + click + | ||
| * selection — anything that signals user attention. Set to ["click"] for | ||
| * sticky-focus UIs where hover doesn't change the AI's reference point. | ||
| */ | ||
| types?: ChartObservation["type"][] | ||
| } |
Update auto-generated bundle size figures across docs. Adjusts gzip sizes for several entry points (e.g. semiotic/xy 85→86KB, semiotic/ordinal 69→70KB, semiotic/network 63→64KB, semiotic/realtime 90→91KB, semiotic/ai 189→211KB, full semiotic 188→190KB) in README.md, CLAUDE.md, and ai/system-prompt.md to reflect the latest build metrics.
Polish three blog entries: anchored-conversations, charts-that-know-what-theyre-for, and live-conversational-dashboard. Changes include ESLint disable for unescaped entities, numerous copy/grammar edits, minor content clarifications, and a date update. UI/UX tweaks: tooltip/background colors, button style adjustments, pinned chart extents to avoid overlay drift, and small layout/margin changes for preview tiles. charts-that-know... also adds several chart components to the preview map, refactors preview margins/size, and introduces a SameDataDifferentIntent demo (fixed intents + QUARTERLY_KPIS) to illustrate different picks for the same data. Overall these are editorial improvements and small interactive/demo enhancements.
Switch realtime chart capabilities to emit timeAccessor/valueAccessor (and categoryAccessor where relevant) instead of xAccessor/yAccessor so recommender outputs are directly usable by realtime components. Add time-field presence checks for several stream charts and include category accessor discovery for heatmap. Rename ForceDirectedGraph prop to canonical nodeIdAccessor and keep deprecated alias behavior. Tighten ProcessSankey temporal validation and improve messages. Update RealtimeLineChart intent scoring comment and remove unsuitable compare-series fit. Add brush-end to default focus types and adjust its docstring. Update tests to expect the new accessor names and include a ts date field in the test stream fixture.
Pass the optional `audience` option through useChartSuggestions (include in destructuring, pass into suggestion builder, and add to memo dependency list) so suggestions can consider audience. Update ProcessSankey capability to build props dynamically and emit startTimeAccessor/endTimeAccessor when input edges use `start`/`end` field names so suggestions are runnable without manual patching. Change Histogram's default categoryAccessor to coerce non-string category values to strings and fall back to "All" for null/undefined, ensuring the accessor always returns a string.
Hoist the Histogram defaultCategoryAccessor to module scope and use it as the component default so the accessor remains referentially stable across renders (avoids unnecessary re-binning for raw-observation and pre-binned data). In DifferenceChartCapability, normalize pivot keys by converting Date instances to timestamps before using them as Map keys so rows with equal timestamps but different Date object identities pair correctly. Also adjust a comment in useChartFocus to clarify that the "*-end" observation types are mapped back to null in the memo used by the reducer.
Skip nullish and empty-string series keys when computing series totals and buckets. AreaChart: avoid counting null/"" series and only subselect the leading series when a valid leading series was found (and totals exist) so single-series rendering isn't forced. DifferenceChart: add an isValidSeries helper and filter out null/"" series when aggregating totals and building the by-x map to prevent literal "undefined"/"" buckets and match profileData's distinct-count semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a headless conversational interrogation API and a chart capability/suggestion layer under semiotic/ai. Adds useChartInterrogation, data summarization, profileData, suggestCharts, scoreChart, useChartSuggestions, registration APIs, MCP tooling (interrogateChart/suggestCharts) and server integration. Includes many new TypeScript implementations and tests, a large set of chart capability descriptors, docs and examples (blog entries, pages, and README/guide updates), new scorecard scripts and a capability-scorecard JSON, an IMPROVEMENTS roadmap, and minor metadata updates (package.json, changelog entry for 3.6.0, server.json). Enables LLM-driven "chat with the chart" workflows and heuristic chart recommendations.
This pull request introduces comprehensive documentation for two major new AI-powered features: Conversational Interrogation and the Chart Capability Layer, across multiple documentation files. These features enable "chat with the chart" interactions and a heuristic chart suggestion engine, enhancing both user experience and developer extensibility.
The most important changes are:
Conversational Interrogation (
semiotic/ai):useChartInterrogationhook, which enables headless, UI-agnostic conversational interactions with charts. This includes details on API usage, expected context for queries, and how to wire AI-generated annotations into chart components.Chart Capability Layer (
semiotic/ai):Release Information:
3.6.0in theCHANGELOG.mdto reflect the upcoming release containing these features.