[avro] Support decimal logical type on fixed base (#798) - #802
Open
29shivam wants to merge 1 commit into
Open
Conversation
29shivam
requested review from
BentsiLeviav,
chernser,
mshustov and
mzitnik
as code owners
July 31, 2026 17:56
Contributor
Author
|
@chernser ready for review — this covers the |
Confluent's Avro converter maps both bytes- and fixed-based Avro decimals to a Kafka Connect Decimal (a BigDecimal). The connector already validates (validateDataSchema), converts (StructToJsonMap) and serializes (doWriteColValue) a Connect Decimal to a ClickHouse Decimal(P, S) column, so the fixed base is handled the same as the already-supported bytes base. - Add AvroDecimalLogicalTest: models the Connect representation the converter emits for a decimal-on-fixed field and drives it through the convert + RowBinary write path, asserting the encoded bytes match writeDecimal (no live ClickHouse needed). - Promote decimal_fixed_logical.json from the incompatible to the compatible Avro fixtures so the integration test exercises it end to end. - CHANGELOG entry under 1.4.1 (unreleased).
29shivam
force-pushed
the
fix/798-avro-decimal-logical
branch
from
August 3, 2026 00:57
6745069 to
e3c67de
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #798 (part of #726).
Confluent's Avro converter maps both bytes- and fixed-based Avro decimals to a Kafka Connect
Decimallogical type whose value is aBigDecimal. The connector already handles that Connect representation end to end — schema validation (ClickHouseWriter.validateDataSchema), conversion (StructToJsonMap), and RowBinary serialization (doWriteColValue) all treat a ConnectDecimalas a ClickHouseDecimal(P, S). The already-compatibledecimal_bytesfixture confirms the bytes base; this change confirms thefixedbase is handled the same way.Changes:
AvroDecimalLogicalTest— models the ConnectDecimal(withconnect.fixed.size) that the converter emits for a decimal-on-fixedfield, drives it through the convert + write path, and asserts the RowBinary bytes matchBinaryStreamUtils.writeDecimal(...). Runs without a live ClickHouse.decimal_fixed_logical.jsonfromincompatible/tocompatible/so the Avro integration test exercises it end to end.1.4.1 (unreleased).Verification: the feature test passes locally; the full end-to-end path (Confluent Platform → schema registry → Avro converter → ClickHouse) runs via the integration test in CI.
Checklist
Delete items not relevant to your PR: