Commit 38d277e
feat(dictionaries): add structural mismatch detection to check_reality() (#4059)
## Summary
- Implements ENG-2765 step 3: structurally compare dictionaries present
in both the infra map and ClickHouse via `SHOW CREATE DICTIONARY`
- Mismatched dicts are removed from the reconciled map so the subsequent
diff generates an `Added` change, resolved by `CREATE OR REPLACE
DICTIONARY` on next apply
- Switched `execute_create_dictionary` from `IF NOT EXISTS` to `OR
REPLACE` to overwrite drifted CH definitions without a manual drop
## Changes
- **`OlapOperations` trait / ClickHouse impl**: new
`show_create_dictionary(db, name)` method
- **`dicts_ddl_equivalent()`**: DDL body comparator that sorts
post-column clauses (PRIMARY KEY, SOURCE, LAYOUT, LIFETIME) to tolerate
ordering differences between our generator and `SHOW CREATE DICTIONARY`
output
- **`check_reality()`**: replaces the `Vec::new()` placeholder with a
full mismatch-detection loop
- **`reconcile_with_reality()`**: removes structurally mismatched dicts
from the reconciled map
## Test plan
- [ ] `cargo test test_dict` — 19 dictionary tests pass (4
DDL-equivalence, 3 detection, 12 existing)
- [ ] `cargo test test_reconcile` — 11 reconciliation tests pass (2 new
dict tests)
- [ ] `cargo clippy --all-targets -- -D warnings` — clean
🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Medium Risk**
> Adds new dictionary DDL introspection/comparison logic and changes
dictionary creation to `CREATE OR REPLACE`, which can overwrite existing
definitions; parsing edge-cases or false positives could trigger
unexpected replaces.
>
> **Overview**
> Adds **structural mismatch detection for ClickHouse dictionaries**
during `check_reality()` by fetching `SHOW CREATE DICTIONARY` and
comparing it against infra-map generated DDL (with whitespace
normalization, clause reordering tolerance, LIFETIME short/long-form
normalization, and ClickHouse Cloud `[HIDDEN]` secret substitution).
>
> Updates reconciliation so **mismatched dictionaries are removed from
the reconciled map**, causing the next diff to recreate them, and
switches dictionary creation to use **`CREATE OR REPLACE DICTIONARY`**
for both create/replace paths to overwrite drifted definitions without
manual drops.
>
> Extends the `OlapOperations` trait + ClickHouse implementation with
`show_create_dictionary()`, and refactors tests by introducing shared
`test_helpers` (mock OLAP client now supports per-dictionary DDL) plus
new unit tests covering DDL equivalence and mismatch/reconcile behavior.
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
00010bf. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent ea2c68e commit 38d277e
6 files changed
Lines changed: 1068 additions & 471 deletions
File tree
- apps/framework-cli/src
- framework/core
- infrastructure/olap
- clickhouse
0 commit comments