fix: protect SQLAlchemy reflection from String bytes format

[sankalpsthakur]

Summary

Supersedes #923 (same change, rebased onto current main — resolves the cursor.py conflict with #927's Cursor.description probe hardening by keeping _leading_keyword and the best-effort DatabaseError guard while threading query_formats through both the primary query and the introspection re-query).

Fixes #920.

set_default_formats("String", "bytes") (or set_read_format("String", "bytes")) is a global read format that makes String columns decode as bytes. The core driver already protects orchestration queries via _INTERNAL_QUERY_FORMATS = {"String": "string"}, but the SQLAlchemy reflection path (cc_sqlalchemy inspector + dialect metadata methods) issued ordinary DB-API queries and was unprotected.

Under the global bytes format this caused:

1. Loud — Inspector.get_columns() / Table(..., autoload_with=...) raised TypeError: a bytes-like object is required, not 'str' on row.type.replace("\n", "").
2. Silent — get_table_names() returned [b'my_table'] instead of ['my_table'], and engine/comment comparisons like row.engine == "Dictionary" failed.

Changes

• Thread per-query query_formats through dbapi.Cursor.execute / executemany and the SQLAlchemy dialect do_execute* methods, mirroring the existing settings / execution_options plumbing.
• Apply the same _INTERNAL_QUERY_FORMATS override to all SQLAlchemy metadata queries (DESCRIBE TABLE, system.tables, SHOW TABLES / SHOW DATABASES, dictionary create SQL, etc.) via with_internal_query_formats(...).
• User SELECTs are unchanged and still honor the global bytes format.

CHANGELOG entry (for UNRELEASED → Bug Fixes)

SQLAlchemy reflection (get_columns, get_table_names, Table(..., autoload_with=...), and related inspector methods) no longer breaks when a global String read format such as set_default_formats("String", "bytes") is configured. Metadata queries now use the same internal String → string query format override as the core driver's orchestration path, so schema cells decode as str while user queries still honor the global format. Closes #920.

(The entry is deliberately not in the diff — the UNRELEASED section conflicts on every upstream merge; happy to add it to CHANGELOG.md on request.)

Checklist

• Unit and integration tests covering the common scenarios were added
• A human-readable description of the changes was provided to include in CHANGELOG
• For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

AI disclosure

This change was drafted with AI assistance and reviewed by the author before submission.

AI/LLM disclosure

• AI coding tools (including Grok and/or Codex agent-assisted editing) were used to help draft or modify code and this PR description.
• I reviewed the complete change, understand the reasoning, and ran the reported local tests before submitting.
• This submission is original work of authorship under the project CLA / contributor terms; AI output was not pasted unreviewed.

[joe-clickhouse]

Hey @sankalpsthakur. Thanks for this contribution as well. Again, I applied the follow-up review fixes directly on this branch, including:

• Statement query formats now take precedence over broader connection and engine wildcard patterns.
• Alembic's startup current database lookup now uses the internal String format override.

Worth noting that the remaining Alembic version-table read limitation is clearly in the changelog and remains outside this PR's scope.

I also merged the latest main in and reconciled conflicts. Thanks again!

[Copilot]

Pull request overview

This PR fixes a ClickHouse SQLAlchemy reflection failure mode when users enable a global String -> bytes read format by ensuring dialect/inspector metadata queries always apply the driver’s internal {"String": "string"} query-format override, while keeping ordinary user queries honoring the global bytes format.

Changes:

• Thread query_formats through the DB-API Cursor.execute / executemany path and SQLAlchemy dialect do_execute* methods via execution_options.
• Wrap SQLAlchemy reflection and other metadata queries (DESCRIBE, system.tables, SHOW TABLES/DBS, EXISTS ..., etc.) with an internal query-format override helper.
• Add unit + integration test coverage and update docs/CHANGELOG to document the behavior and new knobs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
tests/unit_tests/test_sqlalchemy/test_query_formats.py	Adds unit tests for query_formats plumbing and internal-format application on metadata queries.
tests/unit_tests/test_sqlalchemy/test_alembic.py	Verifies Alembic’s current-database lookup uses internal query formats.
tests/integration_tests/test_sqlalchemy/test_reflect.py	Regression test: reflection remains str under global String -> bytes, while user SELECT still yields bytes.
docs/sqlalchemy.mdx	Documents SQLAlchemy execution_options(query_formats=...) usage.
docs/driver-api.mdx	Documents new DB-API cursor query_formats keyword argument.
clickhouse_connect/dbapi/cursor.py	Adds query_formats passthrough for execute/executemany and the LIMIT 0 metadata probe.
clickhouse_connect/cc_sqlalchemy/inspector.py	Introduces with_internal_query_formats() and applies it to inspector metadata queries.
clickhouse_connect/cc_sqlalchemy/dialect.py	Adds _ch_query_formats() merge logic and forwards formats into cursor execution; wraps dialect metadata queries.
clickhouse_connect/cc_sqlalchemy/alembic/impl.py	Applies internal query formats to Alembic startup currentDatabase() lookup.
CHANGELOG.md	Adds a bug-fix entry describing the reflection behavior under global bytes read format.