Description
ClickHouseType.Write accepts a string for almost every scalar column type — the integer types go through Convert.ToXxx(value, CultureInfo.InvariantCulture), DecimalType/Float32Type/Float64Type likewise, UuidType.ExtractGuid has an explicit new Guid((string)data) branch, and Enum8Type/Enum16Type look the string up as an enum name.
The four date/time types do not. AbstractDateTimeType.CoerceToDateTimeOffset(object) switches over DateOnly / DateTimeOffset / DateTime / OffsetDateTime / ZonedDateTime / Instant and falls through to a bare throw new NotSupportedException() — no message, no mention of the value or the target type. So Date, Date32, DateTime/DateTime32 and DateTime64 are the only scalar types where a string value fails, and when it does the caller gets System.NotSupportedException: Specified method is not supported.
Through InsertBinaryAsync this surfaces as ClickHouseBulkCopySerializationException: Error when serializing data with the message-less NotSupportedException as the inner exception, which gives no hint about which column or value was at fault.
This also blocks Map(Date, V) / Map(DateTime, V) columns for any caller whose map keys are strings — the key reaches MapType.Write -> DateType.Write as a string.
This is the .NET counterpart of ClickHouse/clickhouse-java#3132. Note that the other two halves of that report do not apply here: UInt64Type.Write uses Convert.ToUInt64, which accepts the full unsigned range ("18446744073709551615" -> FF FF FF FF FF FF FF FF) and throws OverflowException on "-1" rather than silently wrapping it; and UuidType already handles strings.
ClickHouse server version
26.8.3.105 (verified end-to-end against a running server; the unit-level repro below needs no server).
Reproduction
Unit level, ClickHouse.Driver.Tests (NUnit), no server required:
using System;
using System.IO;
using ClickHouse.Driver.Formats;
using ClickHouse.Driver.Types;
using NUnit.Framework;
public class StringCoercionTests
{
private static void Write(string clickHouseType, object value)
{
var type = TypeConverter.ParseClickHouseType(clickHouseType, TypeSettings.Default);
using var stream = new MemoryStream();
using var writer = new ExtendedBinaryWriter(stream);
type.Write(writer, value);
}
[Test]
public void StringIsAcceptedByEveryScalarType()
{
// These all pass today
Assert.DoesNotThrow(() => Write("Int32", "42"));
Assert.DoesNotThrow(() => Write("UInt64", "18446744073709551615"));
Assert.DoesNotThrow(() => Write("Decimal(9, 2)", "1.23"));
Assert.DoesNotThrow(() => Write("UUID", "61f0c404-5cb3-11e7-907b-a6006ad3dba0"));
// These all throw NotSupportedException today
Assert.DoesNotThrow(() => Write("Date", "2020-01-01"));
Assert.DoesNotThrow(() => Write("Date32", "2020-01-01"));
Assert.DoesNotThrow(() => Write("DateTime", "2020-01-01 12:34:56"));
Assert.DoesNotThrow(() => Write("DateTime64(3)", "2020-01-01 12:34:56.789"));
}
}
Observed (each of the four date/time writes):
Date <- "2020-01-01" THREW NotSupportedException: Specified method is not supported.
Date32 <- "2020-01-01" THREW NotSupportedException: Specified method is not supported.
DateTime <- "2020-01-01 12:34:56" THREW NotSupportedException: Specified method is not supported.
DateTime64(3) <- "2020-01-01 12:34:56.789" THREW NotSupportedException: Specified method is not supported.
Map(Date, String) <- Dictionary<string, string> { ["2020-01-01"] = "x" }
THREW NotSupportedException: Specified method is not supported.
while for comparison, on the same run:
Int32 <- "42" OK: 2A-00-00-00
UInt64 <- "18446744073709551615" OK: FF-FF-FF-FF-FF-FF-FF-FF
Decimal(9,2) <- "1.23" OK: 7B-00-00-00
UUID <- "61f0c404-..." OK: E7-11-B3-5C-04-C4-F0-61-A0-DB-D3-6A-00-A6-7B-90
String <- "abc" OK: 03-61-62-63
End-to-end through the public API, against http://localhost:8123:
using var client = new ClickHouseClient("Host=localhost;Port=8123");
await client.ExecuteNonQueryAsync("DROP TABLE IF EXISTS str_date");
await client.ExecuteNonQueryAsync("CREATE TABLE str_date (id Int32, d Date, u UInt64, g UUID) ENGINE Memory");
// succeeds
await client.InsertBinaryAsync("str_date", new[] { "id", "u", "g" },
new[] { new object[] { "1", "18446744073709551615", "61f0c404-5cb3-11e7-907b-a6006ad3dba0" } });
// throws
await client.InsertBinaryAsync("str_date", new[] { "id", "d" },
new[] { new object[] { "2", "2020-01-01" } });
Actual: ClickHouse.Driver.Copy.ClickHouseBulkCopySerializationException: Error when serializing data, inner NotSupportedException: Specified method is not supported.
Expected: either the row inserts (parsing the string as a date), or the failure names the column, the value and the reason.
Suggested fix
ClickHouse.Driver/Types/AbstractDateTimeType.cs, CoerceToDateTimeOffset(object) (the switch ending in _ => throw new NotSupportedException()):
- Add a
string s branch that parses with CultureInfo.InvariantCulture — DateTimeOffset.TryParse with DateTimeStyles.AssumeUniversal/AdjustToUniversal when the text carries an offset, otherwise DateTime.TryParse with DateTimeStyles.None so the result is Unspecified and goes through the existing wall-clock-in-column-timezone path in CoerceToDateTimeOffset(DateTime). That keeps a string like "2020-01-01 12:34:56" consistent with an Unspecified DateTime carrying the same wall clock, and matches how Convert.ToXxx(..., InvariantCulture) handles strings for the numeric types. This is purely additive: these inputs throw today, so nothing that currently succeeds changes behaviour.
- Independently of (1), give the fallthrough a message —
throw new NotSupportedException($"Cannot convert {value?.GetType()} to a value for {this}"). The current message-less throw is what turns a wrong-type cell into an unusable ClickHouseBulkCopySerializationException.
Related, same family (separate from the above)
AbstractBigIntegerType.Write (ClickHouse.Driver/Types/AbstractBigIntegerType.cs:37) ends its switch with _ => new BigInteger(Convert.ToInt64(value, CultureInfo.InvariantCulture)), so a string for Int128/UInt128/Int256/UInt256 is silently capped at the Int64 range:
Int128 <- "170141183460469231731687303715884105727" THREW OverflowException: Value was either too large or too small for an Int64.
UInt256 <- "1157920892373161954235709850086879078532699846656405640394575840079131296399 35" (same)
BigInteger.Parse(s, CultureInfo.InvariantCulture) for the string case would cover the full width. Happy to split this into its own issue if preferred.
Link
Upstream report: ClickHouse/clickhouse-java#3132
Description
ClickHouseType.Writeaccepts astringfor almost every scalar column type — the integer types go throughConvert.ToXxx(value, CultureInfo.InvariantCulture),DecimalType/Float32Type/Float64Typelikewise,UuidType.ExtractGuidhas an explicitnew Guid((string)data)branch, andEnum8Type/Enum16Typelook the string up as an enum name.The four date/time types do not.
AbstractDateTimeType.CoerceToDateTimeOffset(object)switches overDateOnly/DateTimeOffset/DateTime/OffsetDateTime/ZonedDateTime/Instantand falls through to a barethrow new NotSupportedException()— no message, no mention of the value or the target type. SoDate,Date32,DateTime/DateTime32andDateTime64are the only scalar types where a string value fails, and when it does the caller getsSystem.NotSupportedException: Specified method is not supported.Through
InsertBinaryAsyncthis surfaces asClickHouseBulkCopySerializationException: Error when serializing datawith the message-lessNotSupportedExceptionas the inner exception, which gives no hint about which column or value was at fault.This also blocks
Map(Date, V)/Map(DateTime, V)columns for any caller whose map keys are strings — the key reachesMapType.Write->DateType.Writeas astring.This is the .NET counterpart of ClickHouse/clickhouse-java#3132. Note that the other two halves of that report do not apply here:
UInt64Type.WriteusesConvert.ToUInt64, which accepts the full unsigned range ("18446744073709551615"->FF FF FF FF FF FF FF FF) and throwsOverflowExceptionon"-1"rather than silently wrapping it; andUuidTypealready handles strings.ClickHouse server version
26.8.3.105(verified end-to-end against a running server; the unit-level repro below needs no server).Reproduction
Unit level,
ClickHouse.Driver.Tests(NUnit), no server required:Observed (each of the four date/time writes):
while for comparison, on the same run:
End-to-end through the public API, against
http://localhost:8123:Actual:
ClickHouse.Driver.Copy.ClickHouseBulkCopySerializationException: Error when serializing data, innerNotSupportedException: Specified method is not supported.Expected: either the row inserts (parsing the string as a date), or the failure names the column, the value and the reason.
Suggested fix
ClickHouse.Driver/Types/AbstractDateTimeType.cs,CoerceToDateTimeOffset(object)(the switch ending in_ => throw new NotSupportedException()):string sbranch that parses withCultureInfo.InvariantCulture—DateTimeOffset.TryParsewithDateTimeStyles.AssumeUniversal/AdjustToUniversalwhen the text carries an offset, otherwiseDateTime.TryParsewithDateTimeStyles.Noneso the result isUnspecifiedand goes through the existing wall-clock-in-column-timezone path inCoerceToDateTimeOffset(DateTime). That keeps a string like"2020-01-01 12:34:56"consistent with anUnspecifiedDateTimecarrying the same wall clock, and matches howConvert.ToXxx(..., InvariantCulture)handles strings for the numeric types. This is purely additive: these inputs throw today, so nothing that currently succeeds changes behaviour.throw new NotSupportedException($"Cannot convert {value?.GetType()} to a value for {this}"). The current message-less throw is what turns a wrong-type cell into an unusableClickHouseBulkCopySerializationException.Related, same family (separate from the above)
AbstractBigIntegerType.Write(ClickHouse.Driver/Types/AbstractBigIntegerType.cs:37) ends its switch with_ => new BigInteger(Convert.ToInt64(value, CultureInfo.InvariantCulture)), so a string forInt128/UInt128/Int256/UInt256is silently capped at theInt64range:BigInteger.Parse(s, CultureInfo.InvariantCulture)for the string case would cover the full width. Happy to split this into its own issue if preferred.Link
Upstream report: ClickHouse/clickhouse-java#3132