Skip to content

Conversation

@Synicix
Copy link
Collaborator

@Synicix Synicix commented Nov 8, 2025

Replacement for arrow-digest, since its hash depends on the order of fields elements, and missing some additional data type support that we may need in the future.

Other fixes:

  • For certain types like decimal, timeunit, arrow-digest doesn't hash the meta info about the bytes leading to possible collision between two different data but same byte representation. i.e 1.20 (scale: 2, precision: 3), vs 12.0 (scale:1, precision 3) both have the byte representation of the number 120 in arrow, since scale and precision is stored in metadata.

Fixes PLT-451

@Synicix Synicix marked this pull request as ready for review November 13, 2025 10:51
@Synicix Synicix requested a review from eywalker November 13, 2025 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant