feat: when-then broadcasting #2663
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
camriddell
force-pushed
the
feat-whenthen-broadcast
branch
from
023a936 to
9607f7b
Compare
|
Rebased on top of #2662 do not merge until that PR lands in main. |
|
thanks @camriddell ! could you rebase again please? |
camriddell
force-pushed
the
feat-whenthen-broadcast
branch
from
9607f7b to
2f9edfc
Compare
narwhals/_dask/namespace.py
Outdated
|
|
||
| if self._otherwise_value is None: | ||
| return [then_series.where(condition)] | ||
| otherwise_value = get_dask_expr()._expr.Where._defaults["other"] |
There was a problem hiding this comment.
this is the only part that jumps out to me as suspicious 🤔
MarcoGorelli
approved these changes
Jun 12, 2025
There was a problem hiding this comment.
thanks @camriddell ! really solid work here! really impressed you figured out all the details, and I'm excited that we can support nw.when(nw.col('a').count()>0).then(nw.col('a').sum()) now 🥳
I just made two updates:
- not using private methods from Dask. In general, doing that kind of thing gives a very high chance of future failures, i'd advising against doing that whenever possible
- replace a couple of
if x is Truewith justif x(unless there was some reason i missed for usingif x is True?
This was referenced Jun 12, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this? (check all applicable)
Related issues
Checklist
If you have comments or can explain your changes, please do so below
This adds broadcasting behavior to the
when/thenpattern similar to Polars (see #2645) for our eager backends and dask.The implementation for the Lazy backends will require #2652 unless we want to introduce some hacky workarounds (e.g. we may be able to do this by joining after an aggregation to "broadcast" results; however I haven't fully thought through how this would look in the codebase).