feat(query): cache parameterized logical plans

[dbsid]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR extracts the planner-cache work from #20144. It does not include ordered top-k, PresortedMerge, page statistics, or their settings.

It adds a bounded two-level logical planner cache:

1. The exact-SQL cache stores an optimized concrete logical plan.
2. The parameterized cache shares a bound logical template across predicate literals. On a template hit, the current typed literals are injected and the optimizer runs again.

Only logical planning is cached. Physical planning, block pruning, and execution are performed for every query, so different predicate values still produce value-specific physical plans and scan ranges. The query-result-cache key continues to use the current query's original formatted AST.

Parameterized predicates cover comparisons, IN, BETWEEN, LIKE/regexp, and IS DISTINCT FROM. LIMIT and OFFSET remain part of the template shape and are not parameterized. Unsupported plans fall back to the normal AST bind path. User-written calls using the internal parameter-marker name disable parameterization defensively.

Cache isolation and validation include tenant, catalog, database, optimizer-relevant settings, variables, security context, table IDs/sequences/schemas/snapshots, and security policies. A table-snapshot mismatch evicts the local stale entry before reuse, including when the metadata change was made by another query node. Table functions, external locations, async functions, and other unsafe dependencies are not cached.

Successful DML invalidates entries for affected table IDs. Successful catalog DDL advances the DDL generation and clears the cache, including table/database/index/policy/tag/sequence/dictionary paths. Mutation generations close concurrent insert/invalidate races.

The enable_planner_cache setting supports both session SET and persistent SET GLOBAL scopes. The cache uses a byte-bounded LRU configured by cache.planner_cache_max_bytes, with a 512 MiB default. The previous standalone physical-plan-cache experiment and enable_standalone_fast_planner_cache setting are intentionally not included.

No-lazy benchmark

Authoritative binaries:

• Baseline: 22a7627
• Final: 58ca1fd

Environment: one standalone query node on web3-i4i, on-disk data cache enabled, lazy_read_threshold=0, query-result cache disabled, planner cache enabled, distributed pruning disabled, max_threads=16, 5 concurrent clients, and fixed seed 2026071305.

The test used two interleaved baseline/final rounds. For each process, Q1/Q2/Flex were all warmed first (400 iterations per worker), then each query ran 3,000 measured requests. This preserves mixed-query exact-cache pressure instead of making one query's exact plans artificially hot immediately before measurement.

Query	Mean latency	p95 latency	QPS
Q1	1.99% lower	9.01% lower	2.01% higher
Q2	2.08% lower	9.52% lower	2.12% higher
Flex	2.53% lower	8.50% lower	2.60% higher

All 36,000 measured requests succeeded. Each baseline round recorded 15,030 accesses / 1,103 misses / 13,927 hits. Each final round recorded 15,630 accesses / 603 misses / 15,027 hits; the additional template lookup path produced 597 hits and only 3 misses across the 600 distinct exact-SQL misses.

A query-local hot-cache control warmed and measured each query consecutively. All mean/p95/QPS changes stayed within 0.6%, showing no material regression when every exact plan already fits and is hot.

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Validation performed on web3-i4i:

• cargo fmt --all and format check
• git diff --check
• Targeted clippy for databend-common-functions, databend-common-sql, and databend-query --lib with -D warnings
• databend-query --lib cargo check
• Planner-cache unit tests: 10/10 passed
• 20_0020_planner_cache.test: MySQL handler 75/75 passed
• 20_0020_planner_cache.test: HTTP handler 75/75 passed
• Cache disabled vs enabled, deterministic Q1/Q2/Flex results: 600/600 matched, including row order
• Runtime metric sequence verified cross-literal template hits, exact-plan hits, table DML eviction, CREATE/DROP SEQUENCE clearing, session/global disable behavior, and user-marker bypass
• Final phase-separated A/B: 36,000/36,000 requests passed with zero errors

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[github-actions]

🤖 CI Job Analysis (Retry 1)

Workflow: 29381507205

📊 Summary

• Total Jobs: 87
• Failed Jobs: 11
• Retryable: 0
• Code Issues: 11

❌ NO RETRY NEEDED

All failures appear to be code/test issues requiring manual fixes.

🔍 Job Details

• ❌ linux / test_unit: Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone_minio (query, http): Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone_minio (query, hybrid): Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone (duckdb, 4c, http): Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone (query, 4c, hybrid): Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone (duckdb, 4c, hybrid): Not retryable (Code/Test)
• ❌ linux / sqllogic / standalone (query, 4c, http): Not retryable (Code/Test)
• ❌ linux / sqllogic / cluster (query, 4c, hybrid): Not retryable (Code/Test)
• ❌ linux / sqllogic / cluster (duckdb, 4c, http): Not retryable (Code/Test)
• ❌ linux / sqllogic / cluster (query, 4c, http): Not retryable (Code/Test)
• ❌ linux / sqllogic / cluster (duckdb, 4c, hybrid): Not retryable (Code/Test)

---

🤖 About

Automated analysis using job annotations to distinguish infrastructure issues (auto-retried) from code/test issues (manual fixes needed).