Support plotting xarray grids (hvPlot + VegaLite + DeckGL)#1823
Support plotting xarray grids (hvPlot + VegaLite + DeckGL)#1823ghostiee-11 wants to merge 27 commits into
Conversation
After holoviz#1791 landed, loading a NetCDF file and asking the AI to plot it produces a scatter or line chart rather than a 2D heatmap, because: - hvPlotBaseView had no way to pass a value column (C) to hvplot - The AI agents had no awareness the pipeline was backed by gridded xarray data Changes: - Add `C` param (value column) to hvPlotBaseView; wire into df.hvplot() - Add `quadmesh` to the kind selector - New lumen/ai/gridded.py: detect_gridded() returns coordinate/variable metadata when pipeline.source is XArraySQLSource, else None - BaseViewAgent._generate_yaml_spec: call detect_gridded() once, pass result as `gridded` template variable - BaseViewAgent/main.jinja2 + hvPlotAgent/main.jinja2: conditional block guides LLM to use kind='heatmap' with x/y/C for gridded data - hvPlotAgent._extract_spec: drop by/groupby when kind=heatmap Addresses holoviz#1821 (1/3)
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1823 +/- ##
==========================================
+ Coverage 71.22% 71.79% +0.57%
==========================================
Files 198 202 +4
Lines 34163 34844 +681
==========================================
+ Hits 24332 25017 +685
+ Misses 9831 9827 -4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Per review on holoviz#1824: hvPlot uses `C` only for heatmap (matplotlib hexbin convention), and `z` for image, quadmesh, contourf. Exposing the param as `C` for all gridded kinds would silently pass C= to plot kinds that expect z=. Fix: rename the public param `C` to `z` and map internally to C= only when kind='heatmap', otherwise pass through as z=. Updated prompt guidance in BaseViewAgent/main.jinja2 and hvPlotAgent/main.jinja2 to use z= and mention the other gridded kinds. Tests updated to use z= and a new test verifies the z-to-C mapping is conditional on kind.
ghostiee-11
changed the title
|
I suppose heatmap isn't the right type for plotting gridded data; the better one is hvplot.image or hvplot.quadmesh, and I wonder if there's a way to detect if isinstance(XArraySQLSource) or something. |
Per review on holoviz#1823: heatmap treats x/y as ordinal categories, which is semantically wrong for continuous-coordinate gridded data. quadmesh and image preserve the continuous axes and are the right default for xarray- derived grids. But hvPlot's quadmesh/image/contourf kinds raise NotImplementedError on long-form pandas DataFrames (they expect a 2D array or xarray), and XArraySQLSource returns pandas via SQL. Fix: hvPlotBaseView.get_plot now pivots the long-form DataFrame to a 2D xarray DataArray (via set_index([y, x])[z].to_xarray()) when kind is in _GRIDDED_KINDS = (quadmesh, image, contourf) and x/y/z are set. Safety guards in _to_gridded: - Skips pivot if the input is not a pandas DataFrame (xarray objects pass through unchanged, so future xarray-returning sources work). - Skips pivot if hvplot.xarray failed to import (import flag set at __init__ time; avoids AttributeError at plot time). - Skips pivot if x/y/z are missing or required columns are absent (falls back to hvPlot's own NotImplementedError). - Skips pivot if duplicate (y, x) pairs exist (pivot would be ambiguous). - Raises ValueError if the pivoted grid would exceed _GRIDDED_MAX_CELLS = 10M cells (~80MB for float64) to prevent OOM. Prompt guidance updated: recommends kind='quadmesh' as the primary choice for xarray-derived gridded data, with image/contourf as alternatives and heatmap as a fallback for ordinal/categorical axes. heatmap still stays on the pandas path (it works fine there). hvplot.xarray is imported in hvPlotBaseView.__init__ alongside hvplot.pandas so the .hvplot accessor is available on DataArray. Tests (10 in test_hvplot_gridded.py): - test_hvplot_quadmesh_from_longform_df: renders hv.QuadMesh - test_hvplot_image_from_longform_df: renders hv.Image - test_hvplot_gridded_size_guard_raises: ValueError over cap - test_hvplot_gridded_skips_pivot_if_xarray_unavailable: flag fallback - test_hvplot_gridded_passes_through_non_dataframe: xarray input skip - existing z-param and heatmap tests updated
|
Good call, thanks. Pushed What changed:
Scope choice to flag: the pivot itself is source-agnostic, it fires whenever kind is quadmesh/image/contourf and x/y/z columns are present and unambiguous. That way a plain CSV with lat/lon/value columns also works if the user explicitly picks quadmesh. If you'd prefer to restrict the pivot to isinstance(pipeline.source, XArraySQLSource) only, happy to tighten it. Safety guards in _to_gridded:
43 tests pass in lumen/tests/views/ and lumen/tests/ai/test_gridded.py. |
Pre-commit ruff auto-fix flagged the # noqa as unused; the try/except already silences the ImportError, so the # type: ignore is enough.
The previous lint cleanup over-stripped the noqa on hvplot.dask, which is a genuine F401 (conditional side-effect import for the .hvplot accessor on dask DataFrames). Restored as scoped # noqa: F401 so RUF100 recognises it as a targeted suppression.
Two review responses bundled: 1. Per review on holoviz#1824: BaseViewAgent/main.jinja2 is the base template inherited by hvPlotAgent, VegaLiteAgent, and DeckGLAgent. The previous gridded block named kind='quadmesh'/'image'/'contourf'/'heatmap' which are hvPlot-only concepts. VegaLite uses mark: rect, DeckGL uses layer types, so injecting hvPlot kind values into their prompt context is misleading. Kept the metadata (xarray source, dims, data_vars) in the base block; each agent's own prompt carries the syntax-specific guidance. Updated test to assert no hvPlot kinds leak into the base. 2. Style cleanup for lint discipline: moved MagicMock/patch/holoviews/ numpy/pandas imports to the top of test_hvplot_gridded.py (was inline inside test bodies). xarray is an optional dep, so it is guarded with pytest.importorskip inside the tests that actually use it to keep test-core collection working. Replaced two em dashes in hvPlotAgent prompt and a test docstring.
cc91fec to
43e7574
Compare
Per review on holoviz#1823: detect_gridded is a small one-function helper; it does not warrant its own module. Moved into lumen/ai/utils.py alongside the other AI helpers and removed the standalone gridded.py. Updated imports in lumen/ai/agents/base_view.py and lumen/tests/ai/test_gridded.py.
|
I'm confused about your screenshots: 
Are those both showing the before? If it's after, why doesn't it match |
Polish-and-clarity items from the 2026-04-30 review pass: * views/base.py: replace per-instance ``type(self)._hvplot_xarray_available`` with a single module-level ``_HVPLOT_XARRAY_AVAILABLE`` constant probed once at import; hoist ``import pandas as pd`` to the top; consolidate the early-return guards in ``_to_gridded``; extract ``_gridded_pivot_blocker`` so a gridded ``kind`` with an unpivotable frame surfaces a clear ``ValueError`` instead of hvPlot's opaque ``NotImplementedError``. * ai/utils.py: rename ``detect_gridded`` to ``gridded_metadata`` (the return type is a dict not a bool); collapse the two early-return guards into one; add a ``regular`` flag computed from coord uniformity (with a datetime64-safe path) so the LLM has a concrete signal for the image-vs-quadmesh choice. * prompts/hvPlotAgent: deterministic kind selection driven by ``gridded.regular``: regular grids prefer ``kind='image'``, irregular grids prefer ``kind='quadmesh'`` and are explicitly told not to use ``image``. * tests: two new failure-signal tests (duplicate-rows and missing-z), ``regular`` coverage for regular / irregular / datetime coords, and branch tests for the prompt's image vs quadmesh wording.
* views/base.py: merge the two early returns in `_to_gridded` into a single `if not isinstance(...) or _gridded_pivot_blocker(...)`, consistent with the consolidation pattern asked for in the earlier review pass. * tests: add `test_hvplot_gridded_xarray_dataset_renders` which feeds an xarray Dataset straight into `get_plot` and asserts the result is a real `hv.QuadMesh`. Standard Lumen sources only emit DataFrames, so the test bypasses the pipeline and passes the Dataset directly to the view.
|
Sorry for the confusion on those, you're right that they didn't actually demonstrate the fix. The original demo used a tiny 4x5 synthetic grid which made the after look mostly uniform and almost identical to the before. I've redone the demo with a real dataset ( Before (main): the same view spec raises a clear 
After (this branch): the view pivots the long-form pandas frame to a 2D xarray DataArray and renders the quadmesh with the proper color scale, matching what 
view = hvPlotView(pipeline=pipe, kind="quadmesh", x="lon", y="lat", z="air")
view.get_panel() |
|
Pushed d99b530 which addresses the two follow-ups from your last pass:
|
Wire gridded metadata into VegaLiteAgent's YAML and Altair spec generators, and add mark:rect heatmap guidance plus an example. xarray derived grids now render as viridis heatmaps instead of empty geographic charts. The agent overrides the base generators, so it computes the gridded metadata itself rather than inheriting it.
Wire gridded metadata into DeckGLAgent's declarative and PyDeck spec generators and add ColumnLayer/HexagonLayer guidance. Guard DeckGLView against frames larger than 250k rows, which reliably exhaust browser memory, raising a clear ValueError instead.
ghostiee-11
changed the title
Resolve conflicts in lumen/ai/utils.py and lumen/ai/agents/vega_lite.py: keep both gridded_metadata and the new normalize_vegalite_spec, and switch gridded_metadata + its test from the removed check_xarray_available to the new try_import_xarray helper.
groupby is valid for an hvplot heatmap (it creates a widget selector), so the agent should not strip it. Removed the heatmap by/groupby drop; the gridded prompt guidance already covers appropriate field usage.
- Move the grid-cell cap and DeckGL row cap to module-level constants (GRIDDED_MAX_CELLS, DECKGL_MAX_ROWS) instead of ALL_CAPS class attributes. - Source the gridded plot kinds from hvplot's HoloViewsConverter._gridded_types via _gridded_kinds() rather than hardcoding the tuple. - Detect xarray with the try_import_xarray helper and import hvplot.xarray lazily where the accessor is used, dropping the module-level try/except flag.
The lazy import only ran on the pivot path, so handing an xarray object straight through left the .hvplot accessor unregistered, which failed on a clean import order (green on macOS/Windows, red on Ubuntu CI). Import it at the top of _to_gridded so both the pivot and passthrough paths register it.
Importing hvplot.xarray pulls in xarray, so it must not run when xarray is unavailable (the core test env). Return the blocked DataFrame before the import; only register the accessor once xarray is confirmed present, i.e. on the pivot or the xarray-passthrough path.
| limited to the 2D scalar-field kinds that map cleanly from x/y/z | ||
| columns (points/dataset/rgb are gridded to hvPlot but need no pivot). | ||
| """ | ||
| from hvplot.converter import HoloViewsConverter |
There was a problem hiding this comment.
Is it necessary to have this nested under a function? I think we import hvplot elsewhere maybe?
| return tuple( | ||
| kind for kind in HoloViewsConverter._gridded_types | ||
| if kind in ("contour", "contourf", "image", "quadmesh") | ||
| ) |
There was a problem hiding this comment.
Isn't this simply doing return ("contour", "contourf", "image", "quadmesh") but in a convoluted way?
|
|
||
| ## Gridded xarray data | ||
| `df` is a long-form grid from an xarray dataset (dimensions: {{ gridded.dims | join(', ') }}; variables: {{ gridded.data_vars | join(', ') }}). | ||
| Use `mark_rect()` for a heatmap: encode `x`/`y` as ordinal coordinate dimensions and `color` as the quantitative value with `alt.Scale(scheme='viridis')`. Do not use geographic projections. |
There was a problem hiding this comment.
Why no geographic projections?
|
Thanks! I think this is close. Can you run through xr.tutorial.open_dataset("air_temperature") example and show a video of it being plotted thru Lumen? |
|
Sure.. |
- promote the gridded-kinds list to a module-level GRIDDED_KINDS constant, dropping the nested hvplot import and the convoluted comprehension - allow groupby in the hvPlot prompt to page extra dimensions into a widget slider (by is still discouraged for a 2D scalar field) - explain in the VegaLite prompt why mark_rect stays ordinal (Vega-Lite cannot project rect marks)
Fold any groupby columns into the pivot index so a grid with an extra dimension (e.g. air_temperature's time axis) renders with a widget slider instead of being rejected as duplicate (lat, lon) rows. The size guard and duplicate check now span the full index. Matches the hvPlot prompt guidance to use groupby for extra dimensions.
Non-reducing hvPlot kinds draw one glyph per row, so rendering a large frame (a gridded xarray source expands to millions of long-form rows) can exhaust the browser tab's memory and freeze it. hvPlotView and the AI's hvPlotUIView now refuse to render past MAX_RENDER_ROWS with a clear message pointing at a SQL LIMIT/aggregation, mirroring DeckGLView's existing cap. Gridded/aggregating kinds and rasterize/datashade are exempt.
|
Ran it through air_temperature That dataset is ~3.87M rows long-form. gridded kinds are fine (pivot to a small grid), but scatter/table would ship every row to the browser and freeze the tab. deck.gl already caps rows for this, so i added the same cap to hvPlotView/hvPlotUIView. now a per-row render past the limit raises a clear "reduce via LIMIT/aggregation" error instead of hanging. gridded/aggregating kinds + rasterize/datashade exempt. Screen.Recording.2026-07-10.at.3.56.05.PM.mov |
|
Wait, I thought we're using xql.to_dataset() already; if not, I think this PR should support that or else gridded datasets won't be usable in Lumen.
Separately, does this include time? |
|
Ohh yup, I forgot the xarray-sql thing, this pr was previous of that release will fix this on top of that |
|
yeah it includes time: 2920 time × 25 lat × 53 lon = 3.87M. that's one row per (time, lat, lon) cell in long form, so kept gridded it's just a (2920, 25, 53) array. |
|
So besides hvplot, I don't think Vegalite and Deckgl can utilize timesteps? If accurate, we should instruct the agent to subset only 1 time slice (or more generally, keep it 2D unless colorby/groupby) |
XArraySQLSource gains a to_dataset method that returns a gridded xr.Dataset via xarray-sql's to_dataset (falling back to a long-form pivot on xarray-sql < 0.3, and passing dims explicitly since multiple data variables register as multiple tables). Pipeline.get_dataset exposes it for a plain xarray-backed pipeline. hvPlotView renders gridded kinds straight from that Dataset, and the AI explorer view (hvPlotUIView) now uses hvPlot's grid explorer so image and quadmesh work instead of erroring on long-form pandas. Bumps the xarray extra to xarray-sql>=0.3.1.
VegaLite specs and DeckGL layers are static and cannot page a dimension the way hvPlot's groupby slider can, so a gridded source with an extra dim (e.g. time) must be reduced to 2D. gridded_metadata now splits dims into spatial (x/y) and extra dims, and subset_gridded_to_2d pins each extra dim to its first value before the VegaLite/DeckGL agents render, deterministically (the agents emit a spec over the pipeline and cannot reduce its rows via a prompt). The prompts steer x/y to the spatial axes, and DeckGL getPosition now uses those instead of dims[0]/dims[1], which mapped time onto a positional axis.
Screen.Recording.2026-07-11.at.2.22.22.AM.mp4 |
I wonder why the latitudes are flipped
|
gridded_metadata no longer labels which dims are spatial: a grid can be a lon/lat map, a lon/time Hovmoller, or a lat/level section, so guessing lon/lat by name is wrong. The DeckGL and VegaLite agents now generate the spec first, then subset_gridded_to_2d reads the dims the spec actually references (Vega-Lite encoding fields, deck.gl @@= accessors) and pins only the leftover dims to a single slice. The model picks the axes, so a Hovmoller keeps time while an unused time on a map still collapses. Also slims the gridded prompt blocks to drop the lon/lat prescription and the per-dim filter instructions the code now handles.
| or xarray is unavailable. Prompt guidance alone cannot do this -- the view | ||
| agents emit a spec over the pipeline and do not control its rows -- so the | ||
| reduction has to happen here, deterministically. | ||
| def _spec_field_references(spec: Any) -> set[str]: |
There was a problem hiding this comment.
Should this be more explicit with a package (or something) Literal['vega-lite', 'deckgl'] Literal instead
subset_gridded_to_2d and _spec_field_references now take a kind: Literal['vega-lite', 'deckgl'] instead of sniffing both grammars at once. The Vega-Lite path reads encoding field values and the deck.gl path reads @@= accessor expressions, and each agent passes its own kind.
|
Does Also, can we do a find_all and replace rename |
3 participants
Asking Lumen AI to plot a 2D xarray field (like air temperature over lon/lat) used to fall back to scatter or an empty chart, because the value column had nowhere to go and the agents had no signal the data came from an xarray source. This adds gridded support across hvPlot (quadmesh/image/heatmap), VegaLite (rect heatmap), and DeckGL, so the AI now picks the right 2D encoding automatically. Supersedes #1824 and #1825.
After
Screen.Recording.2026-07-04.at.7.55.18.PM.mov
Closes #1821.