Commit 56ffd5f
authored
# Rationale for this change
The performance of the `add_subtract_columns` function in the
`numerical_utils` module can be improved with a parallel branch of
execution when the project is built with Rayon. The parallel branch also
eliminates the cost of temporarily adding the `Scalar` vectors to the
heap. See note below on why the non-Rayon build needs to keep the
temporary vectors on the heap.
On the Multi-A100 VM with a table size of `1,000,000`, the current
implementation took `11.83`ms in the Sum Count benchmark query.
<img width="1440" height="179" alt="image"
src="https://github.com/user-attachments/assets/06fefdcd-6311-4124-8186-f36fc43eff61"
/>
With this PR, the same call is now `7.78`ms - a `1.52`x improvement
<img width="1440" height="179" alt="image"
src="https://github.com/user-attachments/assets/3eb1ad40-6752-497e-93d0-fec921fe2717"
/>
NOTE, if we eliminate the `Scalar` memory allocation from the original
code and use `scalar_at(i).unwrap()`, the performance regresses.
For example, in the Filter benchmark query, making a temporary copy of
the `Scalars` on the heap took `11.63`ms
<img width="1258" height="120" alt="image"
src="https://github.com/user-attachments/assets/d7b8023b-823c-4ce0-becf-c778d16f43d2"
/>
Updating the code to eliminate the temporary copies of the vectors, and
using `scalars_at(i).unwrap()` took `120.61`ms
<img width="1691" height="125" alt="image"
src="https://github.com/user-attachments/assets/ed3cab66-df43-4ae7-b2f8-01beb71d2255"
/>
Adding an `unsafe_scalar_at` function that doesn't return an `Option`
did not help either, the same section of code using
`unsafe_scalar_at(i)` took `106.31`ms
<img width="1662" height="125" alt="image"
src="https://github.com/user-attachments/assets/99a14ce2-974d-4d28-a65d-3661b5cc199d"
/>
# What changes are included in this PR?
- A parallel branch for Rayon builds is added to the
`add_subtract_columns` function in the `numerical_utils` module
# Are these changes tested?
Yes
1 file changed
+28
-9
lines changedLines changed: 28 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
| 18 | + | |
17 | 19 | | |
18 | 20 | | |
19 | 21 | | |
| |||
27 | 29 | | |
28 | 30 | | |
29 | 31 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
37 | 57 | | |
38 | | - | |
39 | | - | |
| 58 | + | |
40 | 59 | | |
41 | 60 | | |
42 | 61 | | |
| |||
0 commit comments