Commit ebd011c
committed
chore: force extension registration in regexp_extract benchmark
CometBenchmarkBase wires CometSparkSessionExtensions via `withExtensions`,
but that call is silently dropped when `SparkSession.builder.getOrCreate()`
returns an existing session, so the benchmark was running plain Spark in
all four "modes" -- the EXPLAIN plan was just `Project + ColumnarToRow +
FileScan parquet` with no CometScan or CometProject.
Override `getSparkSession` to set `spark.sql.extensions` on the SparkConf
(plus the off-heap and shuffle-manager configs CometTestBase uses) so
Comet planning rules actually fire. The native Rust mode now shows up to
2.5x over Spark on patterns with many matches (e.g. regexp_extract_all /
alternation), and 1.2-1.3x on the simpler shapes.1 parent 1de8c0f commit ebd011c
1 file changed
Lines changed: 31 additions & 0 deletions
Lines changed: 31 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
| 24 | + | |
| 25 | + | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
| |||
52 | 55 | | |
53 | 56 | | |
54 | 57 | | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
55 | 86 | | |
56 | 87 | | |
57 | 88 | | |
| |||
0 commit comments