Skip to content

Binary write: Date/Date32/DateTime/DateTime64 reject string values with a bare NotSupportedException, unlike every other scalar type #614

Description

@claude

Description

ClickHouseType.Write accepts a string for almost every scalar column type — the integer types go through Convert.ToXxx(value, CultureInfo.InvariantCulture), DecimalType/Float32Type/Float64Type likewise, UuidType.ExtractGuid has an explicit new Guid((string)data) branch, and Enum8Type/Enum16Type look the string up as an enum name.

The four date/time types do not. AbstractDateTimeType.CoerceToDateTimeOffset(object) switches over DateOnly / DateTimeOffset / DateTime / OffsetDateTime / ZonedDateTime / Instant and falls through to a bare throw new NotSupportedException() — no message, no mention of the value or the target type. So Date, Date32, DateTime/DateTime32 and DateTime64 are the only scalar types where a string value fails, and when it does the caller gets System.NotSupportedException: Specified method is not supported.

Through InsertBinaryAsync this surfaces as ClickHouseBulkCopySerializationException: Error when serializing data with the message-less NotSupportedException as the inner exception, which gives no hint about which column or value was at fault.

This also blocks Map(Date, V) / Map(DateTime, V) columns for any caller whose map keys are strings — the key reaches MapType.Write -> DateType.Write as a string.

This is the .NET counterpart of ClickHouse/clickhouse-java#3132. Note that the other two halves of that report do not apply here: UInt64Type.Write uses Convert.ToUInt64, which accepts the full unsigned range ("18446744073709551615" -> FF FF FF FF FF FF FF FF) and throws OverflowException on "-1" rather than silently wrapping it; and UuidType already handles strings.

ClickHouse server version

26.8.3.105 (verified end-to-end against a running server; the unit-level repro below needs no server).

Reproduction

Unit level, ClickHouse.Driver.Tests (NUnit), no server required:

using System;
using System.IO;
using ClickHouse.Driver.Formats;
using ClickHouse.Driver.Types;
using NUnit.Framework;

public class StringCoercionTests
{
    private static void Write(string clickHouseType, object value)
    {
        var type = TypeConverter.ParseClickHouseType(clickHouseType, TypeSettings.Default);
        using var stream = new MemoryStream();
        using var writer = new ExtendedBinaryWriter(stream);
        type.Write(writer, value);
    }

    [Test]
    public void StringIsAcceptedByEveryScalarType()
    {
        // These all pass today
        Assert.DoesNotThrow(() => Write("Int32", "42"));
        Assert.DoesNotThrow(() => Write("UInt64", "18446744073709551615"));
        Assert.DoesNotThrow(() => Write("Decimal(9, 2)", "1.23"));
        Assert.DoesNotThrow(() => Write("UUID", "61f0c404-5cb3-11e7-907b-a6006ad3dba0"));

        // These all throw NotSupportedException today
        Assert.DoesNotThrow(() => Write("Date", "2020-01-01"));
        Assert.DoesNotThrow(() => Write("Date32", "2020-01-01"));
        Assert.DoesNotThrow(() => Write("DateTime", "2020-01-01 12:34:56"));
        Assert.DoesNotThrow(() => Write("DateTime64(3)", "2020-01-01 12:34:56.789"));
    }
}

Observed (each of the four date/time writes):

Date          <- "2020-01-01"              THREW NotSupportedException: Specified method is not supported.
Date32        <- "2020-01-01"              THREW NotSupportedException: Specified method is not supported.
DateTime      <- "2020-01-01 12:34:56"     THREW NotSupportedException: Specified method is not supported.
DateTime64(3) <- "2020-01-01 12:34:56.789" THREW NotSupportedException: Specified method is not supported.
Map(Date, String) <- Dictionary<string, string> { ["2020-01-01"] = "x" }
                                           THREW NotSupportedException: Specified method is not supported.

while for comparison, on the same run:

Int32         <- "42"                      OK: 2A-00-00-00
UInt64        <- "18446744073709551615"    OK: FF-FF-FF-FF-FF-FF-FF-FF
Decimal(9,2)  <- "1.23"                    OK: 7B-00-00-00
UUID          <- "61f0c404-..."            OK: E7-11-B3-5C-04-C4-F0-61-A0-DB-D3-6A-00-A6-7B-90
String        <- "abc"                     OK: 03-61-62-63

End-to-end through the public API, against http://localhost:8123:

using var client = new ClickHouseClient("Host=localhost;Port=8123");
await client.ExecuteNonQueryAsync("DROP TABLE IF EXISTS str_date");
await client.ExecuteNonQueryAsync("CREATE TABLE str_date (id Int32, d Date, u UInt64, g UUID) ENGINE Memory");

// succeeds
await client.InsertBinaryAsync("str_date", new[] { "id", "u", "g" },
    new[] { new object[] { "1", "18446744073709551615", "61f0c404-5cb3-11e7-907b-a6006ad3dba0" } });

// throws
await client.InsertBinaryAsync("str_date", new[] { "id", "d" },
    new[] { new object[] { "2", "2020-01-01" } });

Actual: ClickHouse.Driver.Copy.ClickHouseBulkCopySerializationException: Error when serializing data, inner NotSupportedException: Specified method is not supported.
Expected: either the row inserts (parsing the string as a date), or the failure names the column, the value and the reason.

Suggested fix

ClickHouse.Driver/Types/AbstractDateTimeType.cs, CoerceToDateTimeOffset(object) (the switch ending in _ => throw new NotSupportedException()):

  1. Add a string s branch that parses with CultureInfo.InvariantCultureDateTimeOffset.TryParse with DateTimeStyles.AssumeUniversal/AdjustToUniversal when the text carries an offset, otherwise DateTime.TryParse with DateTimeStyles.None so the result is Unspecified and goes through the existing wall-clock-in-column-timezone path in CoerceToDateTimeOffset(DateTime). That keeps a string like "2020-01-01 12:34:56" consistent with an Unspecified DateTime carrying the same wall clock, and matches how Convert.ToXxx(..., InvariantCulture) handles strings for the numeric types. This is purely additive: these inputs throw today, so nothing that currently succeeds changes behaviour.
  2. Independently of (1), give the fallthrough a message — throw new NotSupportedException($"Cannot convert {value?.GetType()} to a value for {this}"). The current message-less throw is what turns a wrong-type cell into an unusable ClickHouseBulkCopySerializationException.

Related, same family (separate from the above)

AbstractBigIntegerType.Write (ClickHouse.Driver/Types/AbstractBigIntegerType.cs:37) ends its switch with _ => new BigInteger(Convert.ToInt64(value, CultureInfo.InvariantCulture)), so a string for Int128/UInt128/Int256/UInt256 is silently capped at the Int64 range:

Int128   <- "170141183460469231731687303715884105727"  THREW OverflowException: Value was either too large or too small for an Int64.
UInt256  <- "1157920892373161954235709850086879078532699846656405640394575840079131296399 35"  (same)

BigInteger.Parse(s, CultureInfo.InvariantCulture) for the string case would cover the full width. Happy to split this into its own issue if preferred.

Link

Upstream report: ClickHouse/clickhouse-java#3132

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions