Is your feature request related to a problem? Please describe.
This is especially helpful when generating larger datasets. This feature will help users tell how much progress has been made per table. And it's something to look at while they wait! 😅
Describe the solution you'd like
Solution: Add an optional progress indicator to tpchgen-cli that displays progress bars while data is being generated.
I'm still reading the up on the the two main crates tpchgen and tpchgen-cli, so I have less specifics in this section, but I think all changes should be scoped to tpchgen-cli crate given the note about dependencies.
Ex: Just the idea
$ tpchgen-cli ... --track or --progress
orders [xxxxxxxxxxxxxx ] 50%
lineitem [x ] 10%
...
At a high level,
- Initialize a progress tracker when a
--track flag is passed
- Hook into existing data generation flow and update the tracker. Somewhere
|
pub async fn generate_in_chunks<G, I, S>( |
I think?
- Mark tables complete when generation finishes
Describe alternatives you've considered
None, open
Additional context
For tpchgen-rs, with ❤️ :
Massive thanks for the work you’ve put into tpchgen-rs. I’ve been spending time reading through the repo to learn more Rust, and the code has been a pleasure to study.
I’m especially looking forward to the upcoming TPC-DS data generator (another gigantic ❤️ @clflushopt) that’s a big part of what motivated me to dig into the project now.
For the terminal display, I have been reading about using indicatif. It is written in Rust and I've been getting more familiar with their API. Right now it's my choice, but open to suggestions here too.
Is your feature request related to a problem? Please describe.
This is especially helpful when generating larger datasets. This feature will help users tell how much progress has been made per table. And it's something to look at while they wait! 😅
Describe the solution you'd like
Solution: Add an optional progress indicator to
tpchgen-clithat displays progress bars while data is being generated.I'm still reading the up on the the two main crates
tpchgenandtpchgen-cli, so I have less specifics in this section, but I think all changes should be scoped to tpchgen-cli crate given the note about dependencies.Ex: Just the idea
At a high level,
--trackflag is passedtpchgen-rs/tpchgen-cli/src/generate.rs
Line 51 in d16d958
Describe alternatives you've considered
None, open
Additional context
For tpchgen-rs, with ❤️ :
Massive thanks for the work you’ve put into tpchgen-rs. I’ve been spending time reading through the repo to learn more Rust, and the code has been a pleasure to study.
I’m especially looking forward to the upcoming TPC-DS data generator (another gigantic ❤️ @clflushopt) that’s a big part of what motivated me to dig into the project now.
For the terminal display, I have been reading about using indicatif. It is written in Rust and I've been getting more familiar with their API. Right now it's my choice, but open to suggestions here too.