Merge pull request #809 from nteract/heatmap-perf

Improve heatmap bin perf

Author: emeeks

CHANGELOG.md

@@ -15,6 +15,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Dev-mode `d.data` access warning** — Frame callbacks (`nodeStyle`, `edgeStyle`, `nodeSize`) now warn in development when users access properties that exist on `.data` but not on the RealtimeNode/RealtimeEdge wrapper (e.g., `d.category` instead of `d.data?.category`). Zero production overhead. Applied to all 5 layout plugins (sankey, force, chord, orbit, hierarchy).
 - **Streaming-first docs narrative** — Landing page and Getting Started page restructured to lead with the streaming engine (push API, two-canvas RAF loop, ring buffer, decay/pulse/staleness/transitions) as the primary differentiator.
 
+### Performance
+
+- **Heatmap scene builder optimized** — Streaming path uses flat `Int32Array`/`Float64Array` grids instead of `Map<string, {data[]}>`, eliminating 50k string key allocations and per-datum array pushes. Static path uses numeric Map keys and precomputed 256-entry color LUT (cached per scheme) instead of per-cell `scaleSequential` calls. Streaming 50k points into 20×20 grid: **0.37ms** (was ~49ms with Map+string approach). Static path ~15% faster at high cardinality.
+
 ### Fixed
 
 - **`@modelcontextprotocol/sdk` removed from production dependencies** — The MCP CLI (`semiotic-mcp`) now bundles the SDK via esbuild, so `npm install semiotic` no longer pulls in the 4MB+ MCP SDK and its transitive deps. The bundled CLI works identically — zero behavior change for `npx semiotic-mcp` users.

benchmarks/unit/render-pipeline.bench.ts

@@ -162,27 +162,56 @@ describe('Scene Builders — Stacked Area', () => {
   })
 })
 
-describe('Scene Builders — Heatmap', () => {
+describe('Scene Builders — Heatmap (static)', () => {
   const ctx = makeCtx({
     config: {
-      heatmapXBins: 20,
-      heatmapYBins: 20,
+      xAccessor: 'x',
+      yAccessor: 'y',
+      valueAccessor: 'value',
     },
   })
 
-  bench('heatmap-1k', () => {
+  bench('heatmap-static-1k', () => {
     buildHeatmapScene(ctx, scatter1k, defaultLayout)
   })
 
-  bench('heatmap-10k', () => {
+  bench('heatmap-static-10k', () => {
     buildHeatmapScene(ctx, scatter10k, defaultLayout)
   })
 
-  bench('heatmap-50k', () => {
+  bench('heatmap-static-50k', () => {
     buildHeatmapScene(ctx, scatter50k, defaultLayout)
   })
 })
 
+describe('Scene Builders — Heatmap (streaming aggregation)', () => {
+  const ctx20 = makeCtx({
+    config: { heatmapAggregation: 'count', heatmapXBins: 20, heatmapYBins: 20, valueAccessor: 'value' },
+  })
+  const ctx50 = makeCtx({
+    config: { heatmapAggregation: 'count', heatmapXBins: 50, heatmapYBins: 50, valueAccessor: 'value' },
+  })
+  const ctx100 = makeCtx({
+    config: { heatmapAggregation: 'mean', heatmapXBins: 100, heatmapYBins: 100, valueAccessor: 'value' },
+  })
+
+  bench('heatmap-stream-10k-20x20', () => {
+    buildHeatmapScene(ctx20, scatter10k, defaultLayout)
+  })
+
+  bench('heatmap-stream-50k-20x20', () => {
+    buildHeatmapScene(ctx20, scatter50k, defaultLayout)
+  })
+
+  bench('heatmap-stream-50k-50x50', () => {
+    buildHeatmapScene(ctx50, scatter50k, defaultLayout)
+  })
+
+  bench('heatmap-stream-50k-100x100-mean', () => {
+    buildHeatmapScene(ctx100, scatter50k, defaultLayout)
+  })
+})
+
 // ── 2. RingBuffer Operations ───────────────────────────────────────────────
 
 describe('RingBuffer — Push Throughput', () => {