Report the CrateDB type reflection could not resolve

[florinutz]

Reflecting a table whose column type the dialect does not know produced sqlalchemy.types.UserDefinedType — the abstract base class, missing get_col_spec(). Reflection appeared to succeed; emitting DDL from the reflected table then died with AttributeError: 'UserDefinedType' object has no attribute 'get_col_spec', naming neither the column nor the type. sys.summits reproduces it: its coordinates column is a geo_point, and the package has exported Geopoint all along without registering it in the map.

Closes #295.

Based on flo/292-numeric-type rather than main, since both branches touch TYPES_MAP and the type compiler, and #292 lands the numeric entry the array derivation here covers as numeric_array. Retarget to main to review the two together.

Root cause

_resolve_type returned TYPES_MAP.get(type_, sqltypes.UserDefinedType). That default is abstract only by a missing method rather than by ABC, so SQLAlchemy instantiates it, and SELECT works because result_processor returns None. Only the type compiler (CreateTable, sa.cast()) reaches the missing method, with a traceback pointing into SQLAlchemy internals.

Arrays were a fixed list of sixteen keys, and CrateDB forms an array type name as <element>_array, so any name off the list fell through the same hole — timestamp without time zone_array among them, while both its scalar and its with time zone sibling were mapped.

Fix

UnresolvedType carries the CrateDB type name reflection could not resolve. Such a column still reads, and CrateTypeCompiler refuses to compile it:

CompileError: (in table 'pg_am', column 'amhandler'):
Unable to represent CrateDB type 'regproc' in SQLAlchemy

The issue proposes sqltypes.NullType, whose only message is "Can't generate DDL for NullType(); did you forget to specify a type on this Column?" — nothing was forgotten, and no CrateDB type is named. UnresolvedType exists to carry that name, and is exported for callers who would rather test isinstance(column.type, UnresolvedType) than skip whole tables.

The refusal lives in the type compiler rather than in get_col_spec, because SQLAlchemy's StrSQLTypeCompiler calls get_col_spec too; str(column.type) keeps working from 1.4 onwards and yields the CrateDB type name.

geo_point and geo_shape map to the Geopoint and Geoshape types already in the package. Array names are derived: strip one _array, resolve the rest, wrap in ARRAY — replacing the fixed list, whose sixteen entries render identically to what derivation produces. Only object_array keeps an entry, for the mutation tracking ObjectArray carries. Two shapes stay unresolved and report the column's own type name: an array whose element is unresolved (ip_array), and an array of arrays, which SQLAlchemy's ARRAY cannot hold. A derived ARRAY converts no values, like the entries it replaces, since sqlalchemy.types.ARRAY has no processors of its own.

Type names arrive from information_schema.columns.data_type, which CrateDB emits lowercase for all 27 distinct values a stock 6.4.1 reports.

Out of scope: reflection still selects only column_name and data_type, so type parameters are lost — a reflected VARCHAR(10) is unbounded, and FLOAT_VECTOR(3) and NUMERIC(10, 2) still refuse DDL for want of theirs.

Tests

tests/reflection_test.py, new — nothing covered TYPES_MAP or _resolve_type before: rendered DDL for every array name the fixed list carried plus the ones derivation adds; every mapped type as the element of its array form, walked over TYPES_MAP itself; refusal naming type, table and column for an unresolvable scalar, an unresolvable element, and arrays two and three deep; printing, selecting and one log record under the column's own type name; live, sys.summits reflected and compiled back to DDL. Green on SQLAlchemy 1.3.24, 1.4.54, 2.0.52 and 2.1.0b3.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c50d045e-5528-4f67-8779-57cd81ccc69e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}