distributed_table and distributed_incremental materializations false-positive raise "cluster setting must be set" when used against a database created with ENGINE = Replicated(...), even when cluster: is correctly configured in the profile.
After #447, on_cluster_clause(...) correctly returns an empty string for Replicated databases — the Replicated database engine broadcasts DDL via ZooKeeper, and ClickHouse rejects ON CLUSTER against it (Code: 80. ON CLUSTER is not allowed for Replicated
database). All other DDL paths in the package (CREATE / DROP / INSERT / ALTER / grants / EXCHANGE / persist_docs / etc.) handle this correctly because they only consult on_cluster_clause's output.
Two leftover guards weren't updated when on_cluster_clause semantics changed:
- dbt/include/clickhouse/macros/materializations/distributed_table.sql:13-16
- dbt/include/clickhouse/macros/materializations/incremental/distributed_incremental.sql:13-16
{% set on_cluster = on_cluster_clause(target_relation) %}
{% if on_cluster.strip() == '' %}
{% do exceptions.raise_compiler_error('To use distributed materialization cluster setting in dbt profile must be set') %}
{% endif %}
These check whether on_cluster_clause returned text, not whether cluster is configured. Post-#447 those are no longer the same condition: cluster can be set and on_cluster_clause legitimately empty (Replicated DB). The guards should consult
adapter.get_clickhouse_cluster_name() instead — that returns None only when cluster: is genuinely unset.
Steps to reproduce
- Connect to a ClickHouse instance and create a Replicated database
- Configure profiles.yml with cluster: and database_engine: Replicated.
- Add any model with {{ config(materialized='distributed_table') }} and run dbt run.
Expected behaviour
Model materializes successfully — _local ReplicatedMergeTree per shard plus a Distributed wrapper, no ON CLUSTER clause anywhere. The Replicated DB engine handles DDL broadcast.
Actual: Compilation Error: To use distributed materialization cluster setting in dbt profile must be set, even though cluster is set.
Code examples, such as models or profile settings
profiles.yml:
my_project:
target: dev
outputs:
dev:
type: clickhouse
host: clickhouse.example.com
port: 443
user: my_user
password: "{{ env_var('CH_PASSWORD') }}"
schema: my_repl_db
secure: true
driver: http
cluster: my_cluster
database_engine: Replicated
custom_settings:
insert_distributed_sync: 1
Minimal model models/sharded.sql:
{{ config(materialized='distributed_table', engine='ReplicatedMergeTree()', sharding_key='rand()', order_by='id') }}
select 1 as id
Suggested fix — replace the guard in both files with:
{% if adapter.get_clickhouse_cluster_name() is none %}
{% do exceptions.raise_compiler_error('To use distributed materialization cluster setting in dbt profile must be set') %}
{% endif %}
or simply delete it: create_distributed_table (called below) already raises 'Cluster name should be defined for using distributed materializations' when cluster is genuinely missing, so the early guard is redundant.
dbt and/or ClickHouse server logs
Compilation Error in model sharded (models/sharded.sql)
To use distributed materialization cluster setting in dbt profile must be set
> in macro materialization_distributed_table_clickhouse (macros/materializations/distributed_table.sql)
> called by model sharded (models/sharded.sql)
Environment
- dbt version: 1.11.11
- dbt-clickhouse version: 1.10.0
- clickhouse-connect version (if using http): 1.1.1
- Python version: 3.13.12
- Operating system: Linux
ClickHouse server
- ClickHouse Server version: 25.10.7.6
- ClickHouse Server non-default settings: target database created with ENGINE = Replicated(...); standard <remote_servers> cluster configuration.
distributed_table and distributed_incremental materializations false-positive raise "cluster setting must be set" when used against a database created with ENGINE = Replicated(...), even when cluster: is correctly configured in the profile.
After #447, on_cluster_clause(...) correctly returns an empty string for Replicated databases — the Replicated database engine broadcasts DDL via ZooKeeper, and ClickHouse rejects ON CLUSTER against it (Code: 80. ON CLUSTER is not allowed for Replicated
database). All other DDL paths in the package (CREATE / DROP / INSERT / ALTER / grants / EXCHANGE / persist_docs / etc.) handle this correctly because they only consult on_cluster_clause's output.
Two leftover guards weren't updated when on_cluster_clause semantics changed:
These check whether on_cluster_clause returned text, not whether cluster is configured. Post-#447 those are no longer the same condition: cluster can be set and on_cluster_clause legitimately empty (Replicated DB). The guards should consult
adapter.get_clickhouse_cluster_name() instead — that returns None only when cluster: is genuinely unset.
Steps to reproduce
Expected behaviour
Model materializes successfully — _local ReplicatedMergeTree per shard plus a Distributed wrapper, no ON CLUSTER clause anywhere. The Replicated DB engine handles DDL broadcast.
Actual: Compilation Error: To use distributed materialization cluster setting in dbt profile must be set, even though cluster is set.
Code examples, such as models or profile settings
Minimal model models/sharded.sql:
Suggested fix — replace the guard in both files with:
or simply delete it: create_distributed_table (called below) already raises 'Cluster name should be defined for using distributed materializations' when cluster is genuinely missing, so the early guard is redundant.
dbt and/or ClickHouse server logs
Environment
ClickHouse server