Skip to content

Conversation

@BoedieJB
Copy link

@BoedieJB BoedieJB commented Oct 7, 2025

Issue?

For one of our kafka topics (Azure Eventhub) we receive a payload where data can be delivered as
{"attribute": null}. Kafka delta ingest currently treats this as "null", when it is an attribute candidate for a string type column.

Why does this happen?

In the coercion tree, this is implicitly cast to a string when the target column for this attribute is of string type.
The logic does not check for nulls, and assumes that a proper null is the string value. This is technically wrong.
Fixed that by explicitly disregard nulls in this coercion.

Result

kdi-null-coersion

We simulated a test where two payloads were sent frequently, one with a proper string, one with a null value.
Here you see the effect post-change, where nulls are now treated as a proper null.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant