Skip to content

GeoDataFrame CRS is dropped through DuckDBSource's WKB roundtrip #1904

Description

@ghostiee-11

Summary

DuckDBSource.get() returns a GeoDataFrame, but its CRS is always None, even when the underlying GEOMETRY data is in a known CRS (e.g. EPSG:4326). The geometry is round-tripped through WKB (ST_AsWKB(...) then GeoSeries.from_wkb(...)), and WKB carries no CRS, so it is lost.

This matters now that #1903 routes geometry columns to maps: get_dataframe_schema reports a crs/geographic flag that the map agents use to choose between a basemap map (DeckGL) and a flat 2D plot (hvPlot). With the CRS dropped, geographic (lat/lon) data reads as geographic: False and mis-routes.

Reproduce

from lumen.sources.duckdb import DuckDBSource
from lumen.util import get_dataframe_schema

# a duckdb file whose `countries` table has a GEOMETRY column in EPSG:4326
src = DuckDBSource(uri="geo.db")
gdf = src.get("countries")
print(gdf.crs)  # -> None   (expected EPSG:4326)

props = get_dataframe_schema(gdf)["items"]["properties"]["geometry"]
print(props)    # -> {..., 'crs': None, 'geographic': False}

Cause

DuckDBSource._fetch_df rebuilds geometry with:

df[col] = gpd.GeoSeries.from_wkb(df[col].apply(bytes))

from_wkb defaults to crs=None, and DuckDB's GEOMETRY type is SRID-less, so the CRS cannot be recovered from the fetched bytes.

Suggested fix

Track the CRS at the Lumen level rather than trying to read it back from DuckDB: capture it when the data is ingested (read_geo_file / the GeoDataFrame handed to DuckDBSource already knows it), persist it as table metadata on the source, and re-apply it in _fetch_df via GeoSeries.from_wkb(..., crs=...).

Impact

Found while testing #1903 (geopandas end-to-end). Repro verified locally.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions