Skip to content

Conversation

@hudclark
Copy link
Contributor

Overview

This PR adds support for encoding/decoding the []byte go type as VARBINARY data.

Additional Information

To keep the scope of this PR small, I did not add support for reading VARBINARY values within complex types. This is not a regression, but a gap in the serialization/deserialization code. Timestamps, Numbers, etc also have the same issue - they are read into go types right off of the decoded JSON. This means that a ROW(X'68656C6C6F') is deserialized as a []interface{}{"aGVsbG8="}. I did update the README to document this behavior.

To showcase this behavior, I've added a (skipped) failing test case. Given the behavior is documented, changing it would be a breaking change. Perhaps an option could be added in the future to recursively parse complex types such that this test case passes?

I'm happy to drop the commit with the skipped test case if we feel it adds noise or does not represent the desired state.

Resolves:


Note

I have signed and emailed an individual CLA.

@cla-bot
Copy link

cla-bot bot commented Feb 27, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Comment on lines +115 to +122
}, func(hc *docker.HostConfig) {
hc.Ulimits = []docker.ULimit{
{
Name: "nofile",
Hard: 4096,
Soft: 4096,
},
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trino requires a minimum of 4096 descriptors in order to startup. On my linux machine, I found docker defaults to a limit of 1024.

I've updated the test configuration such that we can be sure the container is configured with reasonable ulimits.

Copy link
Member

@nineinchnick nineinchnick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Once the CLA is cleared, I can merge it. Please remove prefixes from commit messages.

@hudclark hudclark force-pushed the hudclark/support-varbinary branch from 321f328 to d67f2d8 Compare February 27, 2025 17:21
@cla-bot
Copy link

cla-bot bot commented Feb 27, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@hudclark
Copy link
Contributor Author

Appreciate the quick review @nineinchnick! I've removed the commit message prefixes and updated the readme to resolve that strange formatting.

I also want to call out that this might be considered a breaking change for clients who have previously been reading in VARBINARY as strings, then doing base64 decoding client-side.

@nineinchnick nineinchnick added breaking-change Includes changes that would require changes in the application using this driver enhancement New feature or request labels Feb 27, 2025
@hudclark
Copy link
Contributor Author

I will investigate and fix the failing test shortly

@hudclark hudclark force-pushed the hudclark/support-varbinary branch from d67f2d8 to cefa075 Compare February 27, 2025 18:24
@cla-bot
Copy link

cla-bot bot commented Feb 27, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@nineinchnick
Copy link
Member

@cla-bot check

@cla-bot
Copy link

cla-bot bot commented Mar 10, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@cla-bot
Copy link

cla-bot bot commented Mar 10, 2025

The cla-bot has been summoned, and re-checked this pull request!

@hudclark
Copy link
Contributor Author

Hey @nineinchnick, looks like I've been added now trinodb/cla@4053dbd.

Mind trying again? Thanks!

@nineinchnick
Copy link
Member

@cla-bot check

@cla-bot cla-bot bot added the cla-signed label Mar 13, 2025
@cla-bot
Copy link

cla-bot bot commented Mar 13, 2025

The cla-bot has been summoned, and re-checked this pull request!

@nineinchnick nineinchnick merged commit 2a51f32 into trinodb:master Mar 13, 2025
5 checks passed
@nineinchnick nineinchnick changed the title feat: Support VARBINARY Support VARBINARY Mar 13, 2025
@nineinchnick
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking-change Includes changes that would require changes in the application using this driver cla-signed enhancement New feature or request

Development

Successfully merging this pull request may close these issues.

2 participants