Skip to content

feat: make parquet row groups size configurable #158

Merged
alamb merged 23 commits into
clflushopt:mainfrom
kevinjqliu:kevinjqliu/parquet-row-groups
Aug 18, 2025
Merged

feat: make parquet row groups size configurable #158
alamb merged 23 commits into
clflushopt:mainfrom
kevinjqliu:kevinjqliu/parquet-row-groups

Conversation

@kevinjqliu

@kevinjqliu kevinjqliu commented Jul 15, 2025

Copy link
Copy Markdown
Collaborator

@alamb alamb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @kevinjqliu -- I left some suggestions for your review

Comment thread tpchgen-cli/src/main.rs
Comment thread tpchgen-cli/src/parquet.rs Outdated
Comment thread tpchgen-cli/src/parquet.rs Outdated
Comment thread tpchgen-cli/src/parquet.rs Outdated
@alamb

alamb commented Aug 16, 2025

Copy link
Copy Markdown
Collaborator

I am going to take the liberty and polish up this PR and get it ready to merge

Comment thread tpchgen-cli/src/main.rs
Comment thread tpchgen-cli/src/plan.rs
Comment thread tpchgen-cli/src/plan.rs
Comment thread tpchgen-cli/tests/cli_integration.rs
Comment thread tpchgen-cli/tests/cli_integration.rs
Comment thread tpchgen-cli/tests/cli_integration.rs
@alamb alamb requested a review from clflushopt August 16, 2025 12:19
@alamb

alamb commented Aug 16, 2025

Copy link
Copy Markdown
Collaborator

@kevinjqliu and @clflushopt -- I wonder if I can trouble either if you for a review of this PR? It is mostly tests

@alamb alamb mentioned this pull request Aug 16, 2025
5 tasks

@clflushopt clflushopt left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An awesome PR thanks @alamb for pushing this through ! I left some minor nitpicks that are non-blocking (just generic comment improvements). LGTM !

Comment thread tpchgen-cli/src/main.rs Outdated
Comment thread tpchgen-cli/src/main.rs Outdated
Comment thread tpchgen-cli/src/main.rs Outdated
Comment thread tpchgen-cli/src/main.rs
Co-authored-by: @clflushopt <172141496+clflushopt@users.noreply.github.com>
@alamb

alamb commented Aug 18, 2025

Copy link
Copy Markdown
Collaborator

Thank you for the review @clflushopt !

@alamb alamb merged commit d3956a9 into clflushopt:main Aug 18, 2025
7 checks passed
@kevinjqliu kevinjqliu deleted the kevinjqliu/parquet-row-groups branch August 29, 2025 19:08
@kevinjqliu

Copy link
Copy Markdown
Collaborator Author

Thank you both for getting this across the finish line!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] add an option for row group size

3 participants