-
Notifications
You must be signed in to change notification settings - Fork 2.6k
fix: Support varying quantile values per group in group_by aggregation #25606
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
1269250 to
ea1dbcd
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25606 +/- ##
==========================================
+ Coverage 78.35% 81.11% +2.75%
==========================================
Files 1777 1777
Lines 241720 241898 +178
Branches 3085 3085
==========================================
+ Hits 189406 196205 +6799
+ Misses 51517 44896 -6621
Partials 797 797 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
652fb47 to
2fffe21
Compare
|
@orlp @alexander-beedie @ritchie46 @reswqa @c-peters @MarcoGorelli can we get the review here, we have alot of duplicate issues arising for this one. thanks |
8251863 to
9b2f5dc
Compare
py-polars/tests/unit/operations/aggregation/test_aggregations.py
Outdated
Show resolved
Hide resolved
0c543cc to
cf61dd8
Compare
|
OK, I've pushed my changes. 🏓 |
87e08ed to
101917c
Compare
790f349 to
e4c5b7b
Compare
py-polars/tests/unit/operations/aggregation/test_aggregations.py
Outdated
Show resolved
Hide resolved
py-polars/tests/unit/operations/aggregation/test_aggregations.py
Outdated
Show resolved
Hide resolved
py-polars/tests/unit/operations/aggregation/test_aggregations.py
Outdated
Show resolved
Hide resolved
|
Consolidated the three separate tests into one parametrized test
|
8c5e4ab to
a9cdec2
Compare
eb2a7b3 to
5f80e32
Compare
Co-authored-by: Claude <noreply@anthropic.com>
Fixes #20951.
When using
group_by()with a quantile parameter that varies per group (e.g.,pl.col.quantile.first()), all groups incorrectly received the same quantile value instead of each group using its own.Reproduction
Cause
AggQuantileExpr::evaluate_on_groups()always calledget_quantile()which evaluates the quantile expression against the full dataframe, returning a single scalar. This worked for literal quantile values but failed when the quantile expression varied per group (e.g.,first()aggregation).Fix
Added
agg_varying_quantilewhich accepts a slice of quantile values (one per group) and computes quantile per group using the existing aggregation helpers.polars-core changes:
agg_helper_idx_on_all_with_idxand_agg_helper_slice_with_idxhelpers that pass the group index to closuresagg_varying_quantile_genericthat iterates over groups with their corresponding quantile valuesagg_varying_quantilemethods toFloat32Chunked,Float64Chunked, integerChunkedArray,Series, andColumnpolars-expr changes:
AggQuantileExpr::evaluate_on_groups()now detects whether the quantile is uniform (literal/scalar) or varies per group, and dispatches to the appropriate path