Map google.protobuf.Struct to JSON in decoder#28295
Map google.protobuf.Struct to JSON in decoder#28295ccampo133 wants to merge 3 commits intotrinodb:masterfrom
Conversation
Map google.protobuf.Struct, Value, and ListValue well-known types to Trino's JSON type in the Kafka protobuf schema parser and decoder, following the same pattern as google.protobuf.Any and Timestamp.
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
|
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla |
Description
Map
google.protobuf.Struct,google.protobuf.Value, andgoogle.protobuf.ListValuewell-known types to Trino'sJSONtype in the Kafka protobuf schema parser and decoder. This follows the same pattern used forgoogle.protobuf.Any(which is already mapped to JSON, per 5ebbd2e).I don't believe any configuration flag is needed like the one used for
google.protobuf.Any. UnlikeAny(whichrequires a
DescriptorProviderto resolve thetype_urlfield),Structis entirely self-describing and always safe to decode as JSON.See the description in #28294 for why this approach is appropriate.
Additional context and related issues
The changes span two modules:
Schema parsing in
ProtobufSchemaParser: InterceptsStruct/Value/ListValuetypes before the recursive type detection and maps them to JSON.Runtime decoding in
ProtobufColumnDecoderandProtobufValueProvider: ConvertsStructDynamicMessageinstances to sorted JSON strings usingJsonFormat.printer(). HandlesStructboth as top-level fields and when nested inside arrays, maps, or row types.The
sorted()helper inProtobufColumnDecoderwas also generalized to handle non-object JSON values (e.g.,"foo"from aValuefield or["foo","bar"]from aListValuefield) by usingObject.classinstead ofMap.classin the Jackson mapper. This is backward-compatible since existing callers (Any,Oneof) always produce JSON objects.Closes #28294
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text: