Nullable Fields and Exceptions

[UncleJosef]

How do I make sure that when adding a nullable field to a table on the server, the connector continues to write data to the table in the old fields?
Is there a possibility that an exception could be thrown if an entry could not be written to the table? Now you need to look for this information in the log in the DEBUG entries.

[mzitnik]

@UncleJosef can you describe the senario with an example

[UncleJosef]

Create table on server 25.*
CREATE TABLE SandboxTest.data1
(
Type Int32,
ID UInt64,
Data String
,CreatedOn DateTime64(7) DEFAULT now('UTC')
,ttt Int32 NULL
)
ENGINE = MergeTree
ORDER BY ID

Datastream job:

// Load schema
TableSchema Date1TableSchema = ClickHouseTestHelpers.getTableSchema(host, port, ClickHouseTestHelpers.isSSL, username, password, database, tableName);

clickHouseClientConfig.setSupportDefault(true); // When true - work Ok
POJOConvertor covidPOJOConvertor = new Data1POJOConvertor();
ElementConverter<Data1POJO, ClickHousePayload> convertorData1 = new ClickHouseConvertor<>(Data1POJO.class, covidPOJOConvertor);
ClickHouseAsyncSink POJOSink = new ClickHouseAsyncSink<>(
convertorData1,
MAX_BATCH_SIZE,
MAX_IN_FLIGHT_REQUESTS,
MAX_BUFFERED_REQUESTS,
MAX_BATCH_SIZE_IN_BYTES,
MAX_TIME_IN_BUFFER_MS,
MAX_RECORD_SIZE_IN_BYTES,
clickHouseClientConfig
);

	DataStream<Data1POJO> PojoStream = s_env.fromElements(
			new Data1POJO(123,1L,"ab"),
			new Data1POJO(123,2L,"bc"),
			new Data1POJO(123,3L,"cd"),
			new Data1POJO(123,4L,"de")
	);

PojoStream.sinkTo(POJOSink);

//
public void instrument(OutputStream out, Data1POJO input) throws IOException {
try {
ZoneId zid1 = ZoneId.of("UTC");
LocalDateTime DateTime = LocalDateTime.now(zid1);
Serialize.writeInt32(out, input.Type, false, false, ClickHouseDataType.Int32, false, "Type");
Serialize.writeUInt64(out, input.ID, false, false, ClickHouseDataType.UInt64, false, "ID");
Serialize.writeString(out, input.Data, false, false, ClickHouseDataType.String, false, "Data");
//Serialize.writeTimeDate64(out, DateTime, false, false, ClickHouseDataType.DateTime64, true, "CreatedOn", 7); // commented out so that the default value in the database is set.
Serialize.writeInt32(out, null, false, true, ClickHouseDataType.Int32, false, "ttt");
} catch (Exception e) {
System.err.println("****" + e.getMessage());
throw new RuntimeException(e);
}

// Write in database (No ERRORS AND WARNINGS): ( --1 --)
0 0 1970-01-01 03:00:00.019660800 23356162
0 65536 2026-01-23 11:27:30 NULL
0 0 1970-01-01 03:00:00.026214400 23421954
0 131072 2026-01-23 11:27:30 NULL

// Must when clickHouseClientConfig.setSupportDefault(false); and uncommented Serialize.writeTimeDate64(out, DateTime, false, false, ClickHouseDataType.DateTime64, true, "CreatedOn", 7); (--2--)
123 2 bc 2026-01-23 11:39:39.283609000 NULL
123 4 de 2026-01-23 11:39:39.291713000 NULL
123 1 ab 2026-01-23 11:39:39.283609000 NULL
123 3 cd 2026-01-23 11:39:39.291713000 NULL

When Must when clickHouseClientConfig.setSupportDefault(false); and commented Serialize.writeTimeDate64(out, DateTime, false, false, ClickHouseDataType.DateTime64, true, "CreatedOn", 7); (--3--)
!!!! records are not added to the table (No ERRORS)
ONLY:
DEBUG com.clickhouse.client.internal.apache.hc.client5.http.wire -- http-outgoing-2 << "HTTP/1.1 500 Internal Server Error[\r][\n]"
WARN com.clickhouse.utils.Utils -- Deciding how to handle exception: com.clickhouse.client.api.ServerException: Code: 33. DB::Exception: (transport error: 500)

WHEN clickHouseClientConfig.setSupportDefault(true); and uncommentd Serialize.writeTimeDate64(out, DateTime, false, false, ClickHouseDataType.DateTime64, true, "CreatedOn", 7); (-- 4 --)
Write in database
0 0 2026-01-23 11:58:38
0 0 1970-01-06 05:10:04.654259300 33554432
0 0 2026-01-23 11:58:38
0 0 2026-01-23 11:58:38
0 131072 2026-01-23 11:58:38
0 0 2026-01-23 11:58:38
0 0 1970-01-04 22:37:33.491481700 33554432
0 0 2026-01-23 11:58:38
0 0 2026-01-23 11:58:38
0 65536 2026-01-23 11:58:38

-- 1 -- The situation is working, the default values should be written to the table, but nonsense is being written.
-- 2 -- The schema parameters are set incorrectly. Default=true, but Default=false. It works correctly.
-- 3 -- There is an error in the converter and an error should pop up.
-- 4 -- The program is written correctly, without crutches, but nonsense is being written.