Skip to content

fix: clear scan destination on NULL for LowCardinality(Nullable(T)) - #1933

Open
polyglotAI-bot wants to merge 1 commit into
mainfrom
polyglot/lowcardinality-nullable-scan-clear
Open

fix: clear scan destination on NULL for LowCardinality(Nullable(T))#1933
polyglotAI-bot wants to merge 1 commit into
mainfrom
polyglot/lowcardinality-nullable-scan-clear

Conversation

@polyglotAI-bot

Copy link
Copy Markdown
Collaborator

Description

Fixes #1932.

LowCardinality(Nullable(T)) columns track NULLs through key index 0, and during parse the wrapper disables its inner Nullable (nullable.enable = false). That means the inner Nullable.ScanRow never runs its NULL branch, so LowCardinality.ScanRow is solely responsible for resetting the scan destination on a NULL element. It did return nil without touching dest, so a *T destination reused across rows kept a stale, non-nil value after scanning a NULL. The bug is invisible when each scan targets a fresh destination — which is why it slipped past existing coverage (e.g. TestIssue751, whose destinations are re-declared per row).

The fix routes the NULL case through the same clearing logic Nullable.ScanRow already uses: a new package-level helper scanNullInto resets the **T pointer destinations and calls sql.Scanner.Scan(nil). LowCardinality.ScanRow now calls it on the NULL branch, and Nullable.ScanRow is refactored to call it too (behavior-preserving), so the two paths can no longer drift.

Changes

  • lib/column/nullable.go: extract the NULL-clearing switch from Nullable.ScanRow into a shared scanNullInto(dest any) error helper (no behavior change).
  • lib/column/lowcardinality.go: ScanRow now calls scanNullInto(dest) on a NULL element instead of returning without clearing dest.
  • tests/issues/issue_1932_test.go: regression test.

Test

TestIssue1932_LowCardinalityNullableScanClearsDest inserts 'hi' then NULL into a LowCardinality(Nullable(String)) column, selects them ordered (ORDER BY c NULLS LAST), and scans both rows through the same reused *string. It asserts the non-null row populates the pointer ("hi") and the NULL row resets it to nil. It fails on main (Expected nil, but got: (*string)(0x...)) and passes with this change.

String is used because it is the only base type creatable under LowCardinality(Nullable(...)) on a default server — numeric / Date / DateTime base types require allow_suspicious_low_cardinality_types, which the suite avoids for ClickHouse Cloud compatibility (see #1206). Because LowCardinality.ScanRow now delegates to the very same scanNullInto helper as Nullable.ScanRow, the other clear-list arms (**int64, **float64, **time.Time, …) remain covered by the existing Nullable tests.

Verified locally against ClickHouse 26.x: the new test, the existing LowCardinality/Nullable tests (tests/, tests/std/), TestIssue751, and the lib/column unit tests all pass with no edits to existing tests.

Pre-PR validation gate

  • Deterministic repro (fails on main, passes on branch)
  • Root cause documented above
  • Fix targets the root cause (clears dest at the responsible site)
  • Regression test fails without the fix, passes with it
  • No existing tests broken or weakened
  • Convention compliance per AGENTS.md / .claude/CLAUDE.md (tests/issues/issue_<n>_test.go, t.Cleanup closes the conn + drops the table, version-gated)
  • No public API change (scanNullInto is unexported)
  • CHANGELOG.md intentionally not edited — CONTRIBUTING.md states it is generated automatically at release time

LowCardinality tracks NULLs through key index 0 and disables its inner
Nullable during parse, so LowCardinality.ScanRow — not the base column —
must reset the scan destination on a NULL element. It returned without
touching dest, so a *T destination reused across rows kept a stale,
non-nil value after a NULL (a fresh destination per row masked the bug).

Share the NULL-clearing logic with Nullable.ScanRow via a new scanNullInto
helper and call it from LowCardinality.ScanRow.

Fixes: #1932
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LowCardinality(Nullable(T)): scanning a NULL does not clear a reused pointer destination (stale value)

1 participant