Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testFilters is not using the correct ColumnStatistics #11116

Open
yingsu00 opened this issue Sep 27, 2024 · 1 comment
Open

testFilters is not using the correct ColumnStatistics #11116

yingsu00 opened this issue Sep 27, 2024 · 1 comment
Labels
bug Something isn't working triage Newly created issue that needs attention.

Comments

@yingsu00
Copy link
Collaborator

Bug description

SplitReader uses file level ColumnStatistics to test the filters to eliminate unqualified splits. This uses HiveConnectorUtil::testFilters(). This function get the ColumnStatistics using

const auto columnStats = reader->columnStatistics(typeWithId->id());

reader->columnStatistics() needs to take the column Id, not the id in typeWithId. The id in typeWithId is the traversal sequence number in the type tree, not the index of the columns.

System information

N/A

Relevant logs

No response

@yingsu00 yingsu00 added bug Something isn't working triage Newly created issue that needs attention. labels Sep 27, 2024
@Yuhta
Copy link
Contributor

Yuhta commented Oct 1, 2024

It's a bug in the comment, this should be the traversal ID so that subfield can also be filtered in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage Newly created issue that needs attention.
Projects
None yet
Development

No branches or pull requests

2 participants