Skip to content

LowCardinality(Nullable(T)): scanning a NULL does not clear a reused pointer destination (stale value) #1932

Description

@polyglotAI-bot

Summary

When scanning a NULL element from a LowCardinality(Nullable(T)) column into a reused pointer destination (*T), the driver leaves the destination pointing at the previous row's value instead of setting it to nil. Scanning the same column via Nullable(T) (without the LowCardinality wrapper) clears the destination correctly, so the two disagree.

Reproduction

Against a default server (no special settings needed for the String base type):

ctx := context.Background()
conn, _ := clickhouse.Open(&clickhouse.Options{Addr: []string{"127.0.0.1:9000"}})

conn.Exec(ctx, "CREATE TABLE t (c LowCardinality(Nullable(String))) ENGINE Memory")
conn.Exec(ctx, "INSERT INTO t VALUES ('hi'), (NULL)")

rows, _ := conn.Query(ctx, "SELECT c FROM t ORDER BY c NULLS LAST")
var p *string // reused across rows
for rows.Next() {
    _ = rows.Scan(&p)
    if p == nil {
        fmt.Println("<nil>")
    } else {
        fmt.Printf("%q\n", *p)
    }
}

Observed:

"hi"
"hi"     <-- NULL row: p still points at the previous value

Expected:

"hi"
<nil>    <-- NULL row: p should be reset to nil

Root cause

LowCardinality.ScanRow returns early on a NULL element (key index 0) without touching dest:

// lib/column/lowcardinality.go
func (col *LowCardinality) ScanRow(dest any, row int) error {
	idx := col.indexRowNum(row)
	if idx == 0 && col.nullable {
		return nil // <-- does not clear dest
	}
	return col.index.ScanRow(dest, idx)
}

During parse, LowCardinality disables the inner Nullable (nullable.enable = false) and tracks NULLs itself through key index 0. That means the inner Nullable.ScanRow never runs its NULL branch, so LowCardinality.ScanRow is solely responsible for resetting the destination on NULL — but it doesn't. By contrast, Nullable.ScanRow clears **T pointer destinations and calls sql.Scanner.Scan(nil) for NULLs.

Scope

  • Affects every base type valid under LowCardinality(Nullable(...)). String reproduces on a default server; numeric / Date / DateTime base types require allow_suspicious_low_cardinality_types to create.
  • The bug only manifests when the destination pointer is reused across rows (a non-NULL row followed by a NULL row), which is why it was not caught by existing tests that use a fresh destination per row (e.g. AppendStruct incorrect behavior for LowCardinality(Nullable) fields #751).
  • The native driver path (driver.Conn.Query + Rows.Scan) is affected. The database/sql path is not — it reads columns as driver.Value and clears via convertAssign.

Environment

  • clickhouse-go: main
  • ClickHouse server: 26.x (also applies to older versions)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions