Fix query type detection: a removed comment must still separate tokens - #929
Merged
Merged
Conversation
remove_sql_comments replaced a comment with nothing. The server lexer treats a comment as a token separator, so SELECT/*c*/number FROM numbers(9) is valid SQL, but the classification string became SELECTnumber, the query stopped looking like a SELECT, and the client side query_limit was silently dropped. Likewise a comment before a trailing LIMIT hid it, so the client appended a second LIMIT and the server rejected the query with Code: 62. A removed block comment now leaves a single space behind. A line comment ends at its newline, which is kept, so it already separated its neighbors. Because that space can now land between LIMIT and 0, the columns only probe regex accepts any whitespace there instead of exactly one space, so LIMIT /*c*/0 keeps returning column metadata. Closes #928
polyglotAI-bot
requested review from
joe-clickhouse and
peter-leonov-ch
as code owners
August 3, 2026 21:24
joe-clickhouse
approved these changes
Aug 3, 2026
joe-clickhouse
left a comment
Contributor
There was a problem hiding this comment.
Technically sound. Haven't seen customers hitting this yet but will merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #928
remove_sql_commentsbuilds the string thatQueryContext.is_select,has_limit,is_insert,is_command, theLIMIT 0columns only probe and the DB-API bulk insertcheck all run against. It replaced a removed comment with nothing. The server lexer
treats a comment as a token separator, so
SELECT/*c*/number FROM numbers(9)is validSQL, but the classification string became
SELECTnumber FROM numbers(9), the querystopped looking like a
SELECT, and the client sidequery_limitwas silently notapplied (9 rows came back with
query_limit=2). The same way,SELECT number FROM numbers(9)/*c*/LIMIT 1becamenumbers(9)LIMIT 1,has_limitwasFalse, the client appended its own
LIMIT, and the server rejected the result withCode: 62. Both forms are accepted by the server as written.A
--line comment was never affected: the regex stops before the newline and thenewline is kept, so it separates its neighbors on its own.
Changes
clickhouse_connect/driver/query.py: a removed block comment is replaced with a singlespace. A line comment keeps returning nothing, its terminating newline is the separator.
clickhouse_connect/driver/_backend/httpcommon.py:columns_only_rehard coded asingle space between
LIMITand0. That space can now be the one left by a removedcomment, so
SELECT ... LIMIT /*c*/0would have stopped reaching the columns onlymetadata probe and returned no column names. The regex accepts any whitespace there.
CHANGELOG.md: entry under UNRELEASED.Test
tests/unit_tests/test_driver/test_parser.py::test_remove_comments_separates_tokens:parametrized matrix over the shapes a block comment can take (between two words, before
a trailing
LIMIT, betweenLIMITand0, insideINSERT INTO, multiline, twoadjacent comments, leading and trailing), plus contrast rows pinning the unchanged
behavior of line comments and of a
/* */sequence inside a quoted string or identifier.tests/integration_tests/test_client.py::test_query_with_comment_between_tokens:end to end against a real server with
query_limit=2, for both the sync and the asyncclient via
param_client/call. Fails on unpatched code with 9 rows instead of 2 andwith
Code: 62.tests/integration_tests/test_client.py::test_get_columns_only_with_comment: pins thecolumns only probe for a comment before, inside and after the trailing
LIMIT 0. TheLIMIT /*c*/0row is the regression guard for thecolumns_only_rechange, it passesbefore this PR and would fail with the separator change alone.
test_remove_commentsasserted the exact whitespace of the old output, which encoded themissing separator. Its expectation is updated to the same string with one space per
removed block comment. No other existing test changed.
tests/unit_testsandtests/integration_testsruns match the pre-change baselineon this environment (the only failures are the pre-existing
test_error_handlingandalembic import errors and two unrelated
test_clientfailures, identical with andwithout the patch).
ruff format --check,ruff checkandmypyare clean.Checklist
mainbefore the changeCHANGELOG.mdupdated in the same PRNotes
scanner does not recognize at all. This PR only changes the replacement text for a
comment it does recognize. The two touch the same function, so whichever merges second
needs a small rebase.
weakness in
dbapi/cursor.py::_try_bulk_insert: it takes the table name as everythingup to the first space or paren, so it already mis-parses the legal
INSERT INTO db . tbl VALUES (%s). That is a separate defect and is not touched here.