Skip to content

Add benchmarks crate by copying upstream DataFusion code - #73

Merged
robtandy merged 2 commits into
gabrielmusat/extend-session-builder-traitfrom
tpch
Aug 8, 2025
Merged

Add benchmarks crate by copying upstream DataFusion code#73
robtandy merged 2 commits into
gabrielmusat/extend-session-builder-traitfrom
tpch

Conversation

@gabotechs

@gabotechs gabotechs commented Aug 4, 2025

Copy link
Copy Markdown
Collaborator

Adds TPCH benchmarks by bringing a lot of upstream DataFusion code into this repo.

Steps for running benchmarks:

  1. Generate a TPCH dataset (needs docker installed in the system):
./benchmarks/gen-tpch.sh
  1. Run TPCH benchmarks
cargo run -p datafusion-distributed-benchmarks --release -- tpch --path data/tpch_sf1

For now, this benchmarks do not run distributed, they run as in https://github.com/apache/datafusion

@gabotechs
gabotechs force-pushed the tpch branch 2 times, most recently from 06085b0 to db7f0b0 Compare August 8, 2025 11:39
@gabotechs
gabotechs changed the base branch from main to gabrielmusat/extend-session-builder-trait August 8, 2025 11:40
@gabotechs
gabotechs force-pushed the gabrielmusat/extend-session-builder-trait branch from 82a2c82 to b7b62be Compare August 8, 2025 11:46
@gabotechs
gabotechs force-pushed the tpch branch 2 times, most recently from 7d264f1 to 12ab500 Compare August 8, 2025 11:49
@gabotechs
gabotechs marked this pull request as ready for review August 8, 2025 11:53
@robtandy
robtandy merged commit 5a657a4 into gabrielmusat/extend-session-builder-trait Aug 8, 2025
3 checks passed
@robtandy
robtandy deleted the tpch branch August 8, 2025 15:38
robtandy pushed a commit that referenced this pull request Aug 8, 2025
…text (#85)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend the `SessionBuilder` trait to be able to operate not only at the `SessionStateBuilder` level, but also on `SessionState` and `SessionContext`

* Add benchmarks crate by copying upstream DataFusion code (#73)

* Add benchmarks crate by copying upstream DataFusion code

* Wire-up distributed execution with tpch benchmarks (#86)
robtandy pushed a commit that referenced this pull request Aug 8, 2025
…ture (#84)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend SessionBuilder trait to operate on SessionState and SessionContext (#85)

* Move all test utils to src/ and hide them behind an "integration" feature. This way, other crates and integration tests as well can use it

* Extend the `SessionBuilder` trait to be able to operate not only at the `SessionStateBuilder` level, but also on `SessionState` and `SessionContext`

* Add benchmarks crate by copying upstream DataFusion code (#73)

* Add benchmarks crate by copying upstream DataFusion code

* Wire-up distributed execution with tpch benchmarks (#86)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants