Add column types system, refs #2664 by simonw · Pull Request #2667 · simonw/datasette

simonw · 2026-03-17T03:54:41Z

Implements the column types feature that lets Datasette and plugins annotate
columns with semantic types beyond SQLite storage types (e.g. markdown, email,
url, json, file, point). This enables type-appropriate rendering, validation,
form widgets, and API behavior.

Column types mechanism in Datasette core #2664

Key changes:

New column_types internal DB table for storing assignments
ColumnType dataclass in datasette/column_types.py with render_cell,
validate, and transform_value methods
register_column_types plugin hook for registering types
Built-in url, email, and json column types
Datasette API methods: get/set/remove_column_type(s),
get_column_type_class
Config loading from datasette.json column_types table config key
column_types extra on the table JSON endpoint
Column type info in display_columns extra
Column type render_cell gets priority in rendering pipeline
column_type/column_type_config args added to render_cell hookspec
Write-path validation on insert and update

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

📚 Documentation preview 📚: https://datasette--2667.org.readthedocs.build/en/2667/

Implements the column types feature that lets Datasette and plugins annotate columns with semantic types beyond SQLite storage types (e.g. markdown, email, url, json, file, point). This enables type-appropriate rendering, validation, form widgets, and API behavior. Key changes: - New `column_types` internal DB table for storing assignments - `ColumnType` dataclass in datasette/column_types.py with render_cell, validate, and transform_value methods - `register_column_types` plugin hook for registering types - Built-in url, email, and json column types - Datasette API methods: get/set/remove_column_type(s), get_column_type_class - Config loading from datasette.json `column_types` table config key - `column_types` extra on the table JSON endpoint - Column type info in display_columns extra - Column type render_cell gets priority in rendering pipeline - column_type/column_type_config args added to render_cell hookspec - Write-path validation on insert and update https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

- Add transform_value integration in table JSON endpoint rows - Add tests for: duplicate type name error, row endpoint rendering, transform_value in JSON output, column type priority over plugins, row detail HTML rendering, table HTML rendering, upsert validation, unknown type warning logging, config overwrite on restart, and no-config edge case - Total: 34 column type tests, all passing https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

SQLite allows NULLs in primary key columns by default, so mark database_name, resource_name, and column_name as NOT NULL explicitly. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

- Add register_column_types(datasette) hook documentation with example - Update render_cell signature to include column_type and column_type_config parameters - Fixes test_plugin_hooks_are_documented https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Add documentation for get_column_type, get_column_types, set_column_type, remove_column_type, and get_column_type_class methods on the Datasette instance. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

codecov · 2026-03-17T04:30:41Z

Codecov Report

❌ Patch coverage is 93.68421% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.59%. Comparing base (7f93353) to head (b7578a4).
⚠️ Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
datasette/default_column_types.py	82.25%	11 Missing ⚠️
datasette/app.py	97.72%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2667      +/-   ##
==========================================
+ Coverage   90.50%   90.59%   +0.09%     
==========================================
  Files          53       55       +2     
  Lines        8132     8308     +176     
==========================================
+ Hits         7360     7527     +167     
- Misses        772      781       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw · 2026-03-17T04:52:30Z

datasette/views/database.py

+                column_type=None,
+                column_type_config=None,


These are None because here we are displaying rows in an arbitrary SQL query so we can't (easily) look up column types.

simonw · 2026-03-17T04:54:24Z

datasette/views/table.py

+        else:
+            col_dict["column_type"] = None
+            col_dict["column_type_config"] = None


Moving this into the original col_dict = {...

Set column_type and column_type_config to None in the initial col_dict instead of using an else branch. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw · 2026-03-17T04:57:49Z

datasette/default_column_types.py

+        UrlColumnType(name="url", description="URL"),
+        EmailColumnType(name="email", description="Email address"),
+        JsonColumnType(name="json", description="JSON data"),


I don't like how we have to pass name= and description= here, I want those on the classes.

simonw · 2026-03-17T04:59:10Z

docs/internals.rst

+Returns a ``(column_type_name, config)`` tuple for the specified column. ``column_type_name`` is a string like ``"email"`` or ``"url"``, and ``config`` is a dict or ``None``. If no column type is assigned, returns ``(None, None)``.
+
+.. code-block:: python
+
+    ct_name, config = await datasette.get_column_type(
+        "mydb", "mytable", "email_col"
+    )


I'm not sure the return type of this is right. Can we return an instantiated ColumnType class instead perhaps?

Instead of passing name= and description= as constructor arguments, define them as class attributes on each subclass. This better reflects that they are intrinsic to the type, not configurable per-instance. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw · 2026-03-17T05:03:00Z

docs/plugin_hooks.rst

+register_column_types(datasette)
+--------------------------------
+
+Return a list of :ref:`ColumnType <column_types>` instances to register custom column types. Column types define how values in specific columns are rendered, validated, and transformed.


I think this should return a list of classes, not instances. We can then use instances for column types that have had their config attached to them.

- register_column_types() now returns classes instead of instances - ColumnType.__init__ takes optional config=, baking it into the instance - get_column_type() returns a ColumnType instance (or None) instead of a (name, config) tuple - get_column_types() returns {col: ColumnType instance} instead of tuples - Remove get_column_type_class() - no longer needed - render_cell/validate/transform_value methods no longer take config arg; use self.config instead - render_cell hook takes column_type (ColumnType or None) instead of column_type + column_type_config https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw · 2026-03-17T05:24:11Z

tests/test_column_types.py

+
+    ct = TestType(config={"key": "value"})
+    assert ct.config == {"key": "value"}
+    assert ct.name == "test"


Pointless test.

claude added 7 commits March 17, 2026 02:40

Remove duplicate import logging line

de42696

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Fix linting: remove unused import, apply black formatting

5db4f69

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Add NOT NULL constraints to column_types primary key columns

ad6a020

SQLite allows NULLs in primary key columns by default, so mark database_name, resource_name, and column_name as NOT NULL explicitly. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Document column type internal methods in internals.rst

77bbfb5

Add documentation for get_column_type, get_column_types, set_column_type, remove_column_type, and get_column_type_class methods on the Datasette instance. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Apply black and blacken-docs formatting

9fe10cd

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw commented Mar 17, 2026

View reviewed changes

Move column_type defaults into dict literal

72c8c71

Set column_type and column_type_config to None in the initial col_dict instead of using an else branch. https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw commented Mar 17, 2026

View reviewed changes

claude added 4 commits March 17, 2026 05:18

Update cog-generated plugin list to include default_column_types

da0ea43

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Use 'subclass' instead of 'class' in ColumnType docs

c73a1c9

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

Remove pointless test_column_type_with_config test

b7578a4

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3

simonw commented Mar 17, 2026

View reviewed changes

simonw changed the title ~~Add column types system for semantic column annotations~~ Add column types system, refs #2664 Mar 17, 2026

simonw linked an issue Mar 17, 2026 that may be closed by this pull request

Column types mechanism in Datasette core #2664

Closed

simonw merged commit bc7a19b into main Mar 17, 2026
39 checks passed

simonw deleted the claude/column-types-system-en8tU branch March 17, 2026 05:29

simonw added enhancement plugins internals labels Mar 17, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add column types system, refs #2664#2667

Add column types system, refs #2664#2667
simonw merged 14 commits intomainfrom
claude/column-types-system-en8tU

simonw commented Mar 17, 2026 •

edited

Loading

Uh oh!

codecov bot commented Mar 17, 2026 •

edited

Loading

Uh oh!

simonw Mar 17, 2026

Uh oh!

simonw Mar 17, 2026

Uh oh!

simonw Mar 17, 2026

Uh oh!

simonw Mar 17, 2026

Uh oh!

simonw Mar 17, 2026

Uh oh!

simonw Mar 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

simonw commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

simonw Mar 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

simonw commented Mar 17, 2026 •

edited

Loading

codecov bot commented Mar 17, 2026 •

edited

Loading