feat(sqlalchemy): support materialized CTEs (WITH ... AS MATERIALIZED)

[xoelop]

Closes #900.

ClickHouse inlines a CTE by default, so one referenced more than once has its body executed per reference. 26.3 added WITH <name> AS MATERIALIZED (...), but the dialect had no way to emit it: SQLAlchemy's Select.cte() only takes name, recursive, and nesting.

from clickhouse_connect.cc_sqlalchemy import cte, select

ranked = select(book.c.book_id, ...).where(...).cte("ranked", materialized=True)

# same options for a statement built with the standard sqlalchemy.select
ranked = cte(sa_select(book.c.book_id), "ranked", materialized=True)

The module-level form takes the statement as its first argument, matching the existing final() / prewhere() / limit_by() helpers, and accepts any HasCTE so the dialect's Values CTE support keeps working.

Implementation

The keyword is attached as a CTE prefix scoped to the clickhousedb dialect. SQLAlchemy renders CTE prefixes between the name and the body, exactly where ClickHouse expects the keyword, so no visit_cte override is needed, and a statement shared with another backend compiles unchanged there.

AS NOT MATERIALIZED is not implemented, the server rejects it as a syntax error.

The keyword alone is a silent no-op

The server also needs enable_materialized_cte, and with it off it ignores the keyword rather than erroring. Checked on 26.4.1: HTTP 200, no exception, nothing in system.warnings, and the plan simply inlines the body:

-- enable_materialized_cte=0
Aggregating
  Union
    Expression (...) -> ReadFromSystemNumbers
    Expression (...) -> ReadFromSystemNumbers      -- body read twice

-- enable_materialized_cte=1
MaterializingCTEs (Materialize CTEs before main query execution)
  Aggregating
    Union
      Expression (...) -> ReadFromMemoryStorage
      Expression (...) -> ReadFromMemoryStorage
  MaterializingCTE (Materializing CTE: r) -> ReadFromSystemNumbers   -- body read once

So forgetting the setting costs performance without failing, which the docs now call out explicitly and an integration test pins. The API deliberately does not set it for you; it goes through the normal execution_options(settings=...) path.

Integration tests are gated on min_version("26.3").

Tests

• tests/unit_tests/test_sqlalchemy/test_materialized_cte.py: rendering, default-off, the module-level form on a plain sqlalchemy.select and on a Values construct, TypeError on a statement without CTE support, no leakage to other dialects, the issue's join + IN pattern, distinct compiled-statement cache keys, and recursive + materialized together.
• tests/integration_tests/test_sqlalchemy/test_materialized_cte.py: runs the issue's late-materialization pattern against a real server, parametrized materialized/not, plus the silent-no-op case above.
• Extended tests/type_check/sqlalchemy_select_smoke.py with both entry points.

Docs and CHANGELOG.md updated. Ran locally: unit suite, tests/integration_tests/test_sqlalchemy against 26.4.5, ruff, mypy, and the consumer-install checks (both --strict smoke tests, plus the pyright ratchet, which reports the same count as an untouched main).

🤖 Generated with Claude Code

https://claude.ai/code/session_01VRFJAmiTn7AVS6jaeovP6u

[Copilot]

Pull request overview

Adds ClickHouse SQLAlchemy dialect support for materialized common table expressions by letting users emit WITH <name> AS MATERIALIZED (...) from both the cc_sqlalchemy.select() path and a new module-level helper for plain sqlalchemy.select() statements. This addresses ClickHouse's default CTE inlining behavior (and the associated performance pitfall when a CTE is referenced multiple times) while keeping the keyword scoped to the ClickHouse dialect.

Changes:

• Added materialized: bool = False support to ClickHouseSelect.cte(...), implemented via a dialect-scoped CTE prefix so other backends compile unchanged.
• Added module-level cc_sqlalchemy.cte(statement, name, ..., materialized=...) mirroring Select.cte() for statements built with standard SQLAlchemy constructs (including Values).
• Added unit + integration coverage, type-check smoke updates, docs, and a CHANGELOG.md entry (including the required enable_materialized_cte setting and the server's silent no-op behavior when it is off).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
clickhouse_connect/cc_sqlalchemy/sql/init.py	Implements dialect-scoped MATERIALIZED CTE prefixing and exposes materialized= on ClickHouseSelect.cte() plus a module-level cte() helper.
clickhouse_connect/cc_sqlalchemy/init.py	Re-exports the new cte helper at the package level.
tests/unit_tests/test_sqlalchemy/test_materialized_cte.py	Unit coverage for rendering, scoping to ClickHouse dialect only, cache key separation, and helper behavior (including Values).
tests/integration_tests/test_sqlalchemy/test_materialized_cte.py	Integration coverage against a real server, version-gated to 26.3+, including the “silent no-op without setting” case.
tests/type_check/sqlalchemy_select_smoke.py	Type-check smoke coverage for both entry points returning sa.CTE.
docs/sqlalchemy.mdx	Documents materialized CTE usage, required setting, and dialect scoping behavior.
CHANGELOG.md	Records the user-facing SQLAlchemy feature addition and constraints.

[joe-clickhouse]

@xoelop thanks for the contribution! Everything pretty much looks good. I applied a few follow-ups:

1. I added a client-side check that rejects recursive=True with materialized=True since the server does not support recursive materialized CTEs.
2. Values.cte() requires SQLAlchemy 2.0.42 or later so I added a checkin the tests.
3. I added the new rendering tests to SQLAlchemy 1.4 CI.
4. I documented the experimental setting, analyzer requirement, recursive restriction, and SQLAlchemy version boundary.

Thanks again!