I think the better way to handle this is to ensure that the groupby node we make always has aggs. A grouped aggregation with only keys is just a distinct, so in translate.py when we create the groupby node we can instead do:

if len(aggs) == 0:
    return ir.Distinct(
        schema,
        plc.stream_compaction.DuplicateKeepOption.KEEP_ANY,
        None,
        node.options.slice,
        node.maintain_order,
        ir.Select(schema, keys, False, inp),  # noqa: FBT003
    )

Thanks, that does seem like a nicer spot for it. That turns up two separate issues:

1. Distinct isn't (yet) supported as a multi-partition operation. I gather this is something we'll want to support anyways.
2. The empty agg case already works on branch-25.04 (Though I don't see a test for it. I'll add one). But with the ir.Distinct(...) change there's an issue when grouping with multiple keys, where one is a literal: df.group_by("key", 1).agg()

RuntimeError: CUDF failure at: /home/coder/cudf/cpp/src/table/table_view.cpp:48: Column size mismatch.

I believe something down in distinct is trying to treat the literal as a column that's in the table. I'll keep looking a bit more.

It took some wandering, but the RuntimeError: CUDF failure at: /home/coder/cudf/cpp/src/table/table_view.cpp:48: Column size mismatch. can be fixed by changing the Select that we insert to be ir.Select(schema, keys, True, inp), the only difference being the 3rd argument should_broadcast=True instead of False. I believe that broadcasting is appropriate here but I'll confirm. Either way, this was a useful exercise.

This does still leave the

polars.exceptions.ComputeError: NotImplementedError: Class <class 'cudf_polars.dsl.ir.Distinct'> does not support multiple partitions.

because now the experimental executor is receiving a Distinct expr intead of a GroupBy. @rjzamora IIRC you mentioned you might look into implementing some other operations? LMK if you've already started on that, but I'll get up to speed on that process either way.

now the experimental executor is receiving a Distinct expr intead of a GroupBy. @rjzamora IIRC you mentioned you might look into implementing some other operations? LMK if you've already started on that, but I'll get up to speed on that process either way.

Yeah, I've mostly been stuck trying to expand #17941 to handle "unique" and "n_unique" (which are annoying, because they are Exprs rather than IRs).

For the case of Distinct, it probably makes sense to just lower it to Distinct(..., Shuffle(..., Distinct(...))) (and use simple partition-wise tasks for the Distinct nodes themselves).

It took some wandering, but the RuntimeError: CUDF failure at: /home/coder/cudf/cpp/src/table/table_view.cpp:48: Column size mismatch. can be fixed by changing the Select that we insert to be ir.Select(schema, keys, True, inp), the only difference being the 3rd argument should_broadcast=True instead of False. I believe that broadcasting is appropriate here but I'll confirm. Either way, this was a useful exercise.

Yeah, that's the right thing to do.

I'm seeing some test failures locally that I'll need to look into.

FAILED tests/experimental/test_dataframescan.py::test_parallel_dataframescan[1000] - assert 1 > 1
FAILED tests/experimental/test_groupby.py::test_groupby_raises - Failed: DID NOT RAISE <class 'polars.exceptions.ComputeError'>
FAILED tests/experimental/test_join.py::test_broadcast_join_limit[1] - assert 0 == 2
FAILED tests/experimental/test_join.py::test_broadcast_join_limit[2] - assert 0 == 2
FAILED tests/experimental/test_scan.py::test_parquet_blocksize[2-1000] - assert 1 > 2
FAILED tests/experimental/test_scan.py::test_parquet_blocksize[2-10000] - assert 1 > 2
FAILED tests/experimental/test_scan.py::test_parquet_blocksize[3-1000] - assert 1 > 3
FAILED tests/experimental/test_scan.py::test_parquet_blocksize[3-10000] - assert 1 > 3
FAILED tests/experimental/test_select.py::test_select_reduce_raises - Failed: DID NOT RAISE <class 'polars.exceptions.ComputeError'>
FAILED tests/experimental/test_shuffle.py::test_hash_shuffle - assert 0 == 1

Most likely related to changing the engine test fixtures, but it's not immediately obvious why. One of the fixtures (test_select.py) did use a max partition size of 3 instead of 4. But reverting that doesn't fix those.