map.get with no rows

[akshay-okahu]

During dataframe operations if there are no rows after filtering, and I use map.get, I get a daft.exceptions.DaftCoreException: DaftError::ValueError Need at least 1 series to perform concat exception

The only way I found to avoid the exception is to check df.count_rows() > 0 before using map.get, but count_rows() materialises the dataframe.

Is there a better way to avoid the exception which avoids materialising the dataframe? Is it possible that map.get can avoid throwing the exception in case of no rows

Sample:

import pyarrow as pa
import daft

data = pa.array([[("a", 1)], [("a", 2)]], type=pa.map_(pa.string(), pa.int64()))
table = pa.table({"map_col": data})
df = daft.from_arrow(table)

df = df.where(df["map_col"].map.get("a") == 3)
if df.count_rows() > 0:    # exception is thrown without this check
    df = df.with_column("a", df["map_col"].map.get("a"))
df.show()

Stacktrace:

  File "test/dafttest.py", line 25, in <module>
    df.show()
  File "test/.venv/lib/python3.11/site-packages/daft/api_annotations.py", line 26, in _wrap
    return timed_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "test/.venv/lib/python3.11/site-packages/daft/analytics.py", line 199, in tracked_method
    result = method(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "test/.venv/lib/python3.11/site-packages/daft/dataframe/dataframe.py", line 2891, in show
    dataframe_display = self._construct_show_display(n)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "test/.venv/lib/python3.11/site-packages/daft/dataframe/dataframe.py", line 2848, in _construct_show_display
    for table in get_context().get_or_create_runner().run_iter_tables(builder, results_buffer_size=1):
  File "test/.venv/lib/python3.11/site-packages/daft/runners/native_runner.py", line 90, in run_iter_tables
    for result in self.run_iter(builder, results_buffer_size=results_buffer_size):
  File "test/.venv/lib/python3.11/site-packages/daft/runners/native_runner.py", line 85, in run_iter
    yield from results_gen
  File "test/.venv/lib/python3.11/site-packages/daft/execution/native_executor.py", line 37, in <genexpr>
    return (
           ^
daft.exceptions.DaftCoreException: DaftError::ValueError Need at least 1 series to perform concat

I noticed struct.get doesn't throw the exception:

import daft

df = daft.from_pydict({
    "data": [
        {"a": 1},
        {"a": 2}
    ]
})

df = df.where(df["data"].struct.get("a") == 3)
df = df.with_column("a", df["data"].struct.get("a"))
df.show()

[universalmind303]

this definitely looks like a bug. I opened up an issue and a subsequent PR to address this.