Merge pull request #20 from DeepLcom/chore/merge-main-to-dap-main

Chore/merge main to dap main

Author: canbekley

.github/workflows/test_matrix.yml

@@ -1,7 +1,7 @@
 ---
 name: "test_matrix"
 
-on:  # yamllint disable-line rule:truthy
+on: # yamllint disable-line rule:truthy
   pull_request:
     branches: main
   push:
@@ -12,6 +12,7 @@ on:  # yamllint disable-line rule:truthy
     paths-ignore:
       - '**.md'
       - 'LICENSE'
+  workflow_dispatch:
 
 jobs:
   tests:
@@ -23,17 +24,30 @@ jobs:
     strategy:
       matrix:
         python-version:
-          - '3.10'
-          - '3.11'
           - '3.12'
           - '3.13'
         clickhouse-version:
-          - '25.1'
+          - '25.3'
+          - latest
 
     steps:
       - name: Checkout
         uses: actions/checkout@v4
 
+      - name: Set environment variables
+        if: ${{ matrix.clickhouse-version == '22.3' }}
+        run: |
+          echo "TEST_SETTINGS_FILE=22_3" >> $GITHUB_ENV
+          echo "DBT_CH_TEST_CH_VERSION=22.3" >> $GITHUB_ENV
+
+      # a fix until docker compose v2.36.0 will be the default version in the github runner
+      - name: Install Docker Compose v2.36.0
+        run: |
+          sudo mkdir -p /usr/local/lib/docker/cli-plugins
+          sudo curl -L "https://github.com/docker/compose/releases/download/v2.36.0/docker-compose-linux-x86_64" -o /usr/local/lib/docker/cli-plugins/docker-compose
+          sudo chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
+          docker compose version
+
       - name: Run ClickHouse Cluster Containers
         env:
           PROJECT_ROOT: ${{ github.workspace }}/tests/integration

CHANGELOG.md

@@ -1,3 +1,140 @@
+### Release [1.9.2], 2025-06-03
+
+#### Bugs
+* Limit dbt-core version to <1.10.X to avoid compatibility issues ([#453](https://github.com/ClickHouse/dbt-clickhouse/pull/453))
+* README file was broken and fixed in ([#454](https://github.com/ClickHouse/dbt-clickhouse/pull/454))
+* Snapshots were not worked properly on cluster, fixed in ([#455](https://github.com/ClickHouse/dbt-clickhouse/pull/455))
+* when the last line of a model's SQL query is a comment (-- some comment) and the table's contract is enforced, the last parenthesis of the wrapping subquery ends up commented as well. Was fixed in ([#457](https://github.com/ClickHouse/dbt-clickhouse/pull/457))
+* Check for Shared database engine in can_exchange ([#460](https://github.com/ClickHouse/dbt-clickhouse/pull/460))
+* Tests were broken because of docker compose version `2.35` and fixed in ([#468](https://github.com/ClickHouse/dbt-clickhouse/pull/468))
+
+### Release [1.9.1], 2025-04-28
+
+#### Bugs
+* Fix missing database_engine error ([#450](https://github.com/ClickHouse/dbt-clickhouse/pull/450))
+
+
+### Release [1.9.0], 2025-04-28
+
+#### New Features
+* Added ability to set [SQL Security](https://clickhouse.com/docs/en/sql-reference/statements/create/view#sql_security) for normal views ([#379](https://github.com/ClickHouse/dbt-clickhouse/pull/379)).
+* Add support for "microbatch" incremental strategy ([#404](https://github.com/ClickHouse/dbt-clickhouse/pull/404))
+* Added support for [TTL (time-to-live)](https://clickhouse.com/docs/guides/developer/ttl) as a column configuration for `table` and `ephemeral` materializations. This feature is implemented as a [custom constraint](https://docs.getdbt.com/reference/resource-properties/constraints#custom-constraints), which requires model contracts to be enforced ([#442](https://github.com/ClickHouse/dbt-clickhouse/pull/442))
+For example:
+
+  ```sql
+  -- test_ttl.sql
+  {{ config(order_by='(ts)', engine='MergeTree()', materialized='table') }}
+
+  SELECT now() AS ts, 
+        'Some value that should expire!' AS col_ttl
+  ```
+
+  ```yaml
+  models:
+    - name: test_ttl
+      description: 'Testing column-level TTL'
+      config:
+        contract:
+          enforced: true
+      columns:
+        - name: ts
+          data_type: timestamp
+        - name: col_ttl
+          data_type: String
+          ttl: ts + INTERVAL 1 DAY
+  ```
+
+### Improvements
+* Upgrade `dbt-core` to version `1.9` and `dbt-adapters` to `>=1.10` ([#403](https://github.com/ClickHouse/dbt-clickhouse/pull/403)).
+* Escape column names in comments ([#428](https://github.com/ClickHouse/dbt-clickhouse/pull/428)).
+* TTl is now available for distributed tables ([#430](https://github.com/ClickHouse/dbt-clickhouse/pull/430)).
+* Previously, delete_insert would fall back to legacy silently. Now it raises an error if LWD is not enabled ([#447](https://github.com/ClickHouse/dbt-clickhouse/pull/447)).
+* Refactored the logic for applying the `ON CLUSTER` clause during model creation ([#447](https://github.com/ClickHouse/dbt-clickhouse/pull/447)).
+    - Previously, `ON CLUSTER` was inconsistently applied based on engine type and materialization, often leading to confusion and unexpected results.
+    - Now, if a `cluster` is defined in the profile and the engine is **not** `Replicated`, the `ON CLUSTER` clause will **always** be added by default.
+    - A new model-level config `disable_on_cluster: true` has been introduced to explicitly opt out of this behavior.
+    - ⚠️ **Breaking Change**: This modifies the previous behavior. Users relying on the old implicit logic should review and update their models accordingly.
+
+
+### Release [1.8.9], 2025-02-16
+
+#### Improvements
+* It is now possible to configure a TLS client certificate using `client_cert` and `client_cert_key` profile parameters. ([#413](https://github.com/ClickHouse/dbt-clickhouse/pull/413))
+* Added Support of insert_overwrite in cluster setup with incremental and distributed_incremental materializations ([#394](https://github.com/ClickHouse/dbt-clickhouse/pull/394))
+* Improve index and projections creation process ([#421](https://github.com/ClickHouse/dbt-clickhouse/pull/421)) 
+
+#### Bugs
+* Reverted breaking changes in MV materialization ([#416](https://github.com/ClickHouse/dbt-clickhouse/pull/416))
+* A fix was introduced for distributed tables, where an incremental local table could have been dropped if the distributed table was missing. ([#363](https://github.com/ClickHouse/dbt-clickhouse/pull/363))
+
+
+### Release [1.8.8], 2025-02-05
+### Improvements
+* Materialized view now attempts to use `ALTER TABLE...MODIFY QUERY` to update existing materialized views. This is an atomic operation so data is not lost. ([#390](https://github.com/ClickHouse/dbt-clickhouse/pull/390))
+* Make view materialization updates atomic. ([#412](https://github.com/ClickHouse/dbt-clickhouse/pull/412))
+* Create a black list settings to ignore based on the configured Engine. ([#367](https://github.com/ClickHouse/dbt-clickhouse/pull/367))
+
+#### New Features
+* [ClickHouse indexes](https://clickhouse.com/docs/en/optimize/sparse-primary-indexes) are now fully supported for `table` materialization.
+
+
+#### New Features
+* [ClickHouse indexes](https://clickhouse.com/docs/en/optimize/sparse-primary-indexes) are now fully supported for `table` materialization.
+The index config should be added to the model config. for instance: 
+  ```python
+  {{ config(
+         materialized='%s',
+         indexes=[{
+            'name': 'your_index_name',
+            'definition': 'your_column TYPE minmax GRANULARITY 2'
+         }]
+  ) }}
+  ```
+
+### Bug Fixes
+* Materializations are now compatible with `Replicated` database engine, as they will no longer use `ON CLUSTER` statements.
+
+### Release [1.8.7], 2025-01-05
+
+### New Features
+* Added support for [refreshable materialized view](https://clickhouse.com/docs/en/materialized-view/refreshable-materialized-view) ([#401](https://github.com/ClickHouse/dbt-clickhouse/pull/401))
+
+### Improvement
+* Avoid potential data loss by using `CREATE OR REPLACE DICTIONARY` to atomically update a dictionary ([#393](https://github.com/ClickHouse/dbt-clickhouse/pull/393))
+* Removed support in python 3.8 as it is no longer supported by dbt ([#402](https://github.com/ClickHouse/dbt-clickhouse/pull/402)
+
+### Bug Fixes
+* Fix a minor bug related to validating existence of an old hanging mv ([#396]()) 
+
+### Release [1.8.6], 2024-12-05
+
+### Improvement
+* Today, on mv model creation, the target table is being populated with the historical data based on the query provided in the mv creation. This catchup mechanism is now behind a config flag and enabled by default (as is today). ([#399](https://github.com/ClickHouse/dbt-clickhouse/pull/399))
+
+
+### Release [1.8.5], 2024-11-19
+
+### New Features
+* Added support for the creation of more than one materialized view inserting records into the same target table. ([#360](https://github.com/ClickHouse/dbt-clickhouse/pull/364))
+
+### Improvement
+* Added support for [range_hashed](https://clickhouse.com/docs/en/sql-reference/dictionaries#range_hashed) and [complex_key_range_hashed](https://clickhouse.com/docs/en/sql-reference/dictionaries#complex_key_range_hashed) layouts to the dictionary materialization. ([#361](https://github.com/ClickHouse/dbt-clickhouse/pull/361))
+* Truncated stack trace for database errors for cleaner output when HIDE_STACK_TRACE variable is set to any value. ([#382](https://github.com/ClickHouse/dbt-clickhouse/pull/382))
+* It is now possible to pass query settings not only on table creation but also on query. ([#362](https://github.com/ClickHouse/dbt-clickhouse/pull/362))
+
+
+### Bug Fixes
+* Before this version, `split_part` macro used to add an extra quotation. that was fixed in ([#338](https://github.com/ClickHouse/dbt-clickhouse/pull/338))
+
+### Bug Fixes
+* Existing local tables are no longer dropped/recreated in case of missing distributed tables in `distributed_incremental` materialization mode. ([#363](https://github.com/ClickHouse/dbt-clickhouse/pull/363))
+
+### Release [1.8.4], 2024-09-17
+### Improvement
+* The S3 help macro now support a `role_arn` parameter as an alternative way to provide authentication for S3 based models.  Thanks to
+[Mitchell Bregman](https://github.com/mitchbregs) for the contribution!
+
 ### Release [1.8.3], 2024-09-01
 ### Bug Fixes
 * An [issue](https://github.com/ClickHouse/dbt-clickhouse/issues/348) was detected when using multiple projections. We solved it and added a test to cover that use case. ([#349](https://github.com/ClickHouse/dbt-clickhouse/pull/349))
@@ -522,4 +659,4 @@ for Replicated tables that use the `{uuid}` macro in the path to avoid name conf
 [0.19.1]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0.2...v0.19.1
 [0.19.0.2]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0.1...v0.19.0.2
 [0.19.0.1]: https://github.com/ClickHouse/dbt-clickhouse/compare/v0.19.0...v0.19.0.1
-[0.19.0]: https://github.com/ClickHouse/dbt-clickhouse/compare/eb3020a...v0.19.0
+[0.19.0]: https://github.com/ClickHouse/dbt-clickhouse/compare/eb3020a...v0.19.0