Skip to content

Commit bc39d96

Browse files
Benoit Aubuchonclaude
andauthored
feat(dictionaries): add structural mismatch detection to check_reality() (#4059)
## Summary - Implements ENG-2765 step 3: structurally compare dictionaries present in both the infra map and ClickHouse via `SHOW CREATE DICTIONARY` - Mismatched dicts are removed from the reconciled map so the subsequent diff generates an `Added` change, resolved by `CREATE OR REPLACE DICTIONARY` on next apply - Switched `execute_create_dictionary` from `IF NOT EXISTS` to `OR REPLACE` to overwrite drifted CH definitions without a manual drop ## Changes - **`OlapOperations` trait / ClickHouse impl**: new `show_create_dictionary(db, name)` method - **`dicts_ddl_equivalent()`**: DDL body comparator that sorts post-column clauses (PRIMARY KEY, SOURCE, LAYOUT, LIFETIME) to tolerate ordering differences between our generator and `SHOW CREATE DICTIONARY` output - **`check_reality()`**: replaces the `Vec::new()` placeholder with a full mismatch-detection loop - **`reconcile_with_reality()`**: removes structurally mismatched dicts from the reconciled map ## Test plan - [ ] `cargo test test_dict` — 19 dictionary tests pass (4 DDL-equivalence, 3 detection, 12 existing) - [ ] `cargo test test_reconcile` — 11 reconciliation tests pass (2 new dict tests) - [ ] `cargo clippy --all-targets -- -D warnings` — clean 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Adds new dictionary DDL introspection/comparison logic and changes dictionary creation to `CREATE OR REPLACE`, which can overwrite existing definitions; parsing edge-cases or false positives could trigger unexpected replaces. > > **Overview** > Adds **structural mismatch detection for ClickHouse dictionaries** during `check_reality()` by fetching `SHOW CREATE DICTIONARY` and comparing it against infra-map generated DDL (with whitespace normalization, clause reordering tolerance, LIFETIME short/long-form normalization, and ClickHouse Cloud `[HIDDEN]` secret substitution). > > Updates reconciliation so **mismatched dictionaries are removed from the reconciled map**, causing the next diff to recreate them, and switches dictionary creation to use **`CREATE OR REPLACE DICTIONARY`** for both create/replace paths to overwrite drifted definitions without manual drops. > > Extends the `OlapOperations` trait + ClickHouse implementation with `show_create_dictionary()`, and refactors tests by introducing shared `test_helpers` (mock OLAP client now supports per-dictionary DDL) plus new unit tests covering DDL equivalence and mismatch/reconcile behavior. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 00010bf. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent ea2c68e commit bc39d96

6 files changed

Lines changed: 1068 additions & 471 deletions

File tree

0 commit comments

Comments
 (0)