Closed
Description
What happened?
The polars ArrayJoin operation does not behave like documented
When the array is empty, the expected value is NULL, but the result is an empty string. (ID 3 below).
ibis.set_backend("polars")
table = ibis.memtable(
[
{"id": 1, "arr": ["a", "b", "c"], "expected": "a|b|c"},
{"id": 2, "arr": None, "expected": None},
{"id": 3, "arr": [], "expected": None},
{"id": 4, "arr": ["b", None], "expected": "b"},
]
)
table.mutate(
result=_["arr"].join("|"),
).preview()
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┓
┃ id ┃ arr ┃ expected ┃ result ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━┩
│ int64 │ array<string> │ string │ string │
├───────┼────────────────────┼──────────┼────────┤
│ 1 │ ['a', 'b', ... +1] │ a|b|c │ a|b|c │
│ 2 │ NULL │ NULL │ NULL │
│ 3 │ [] │ NULL │ ~ │ <-- issue here
│ 4 │ ['b', None] │ b │ b │
└───────┴────────────────────┴──────────┴────────┘
Proposed change for code:
@translate.register(ops.ArrayStringJoin)
def array_string_join(op, **kw):
arg = translate(op.arg, **kw)
sep = _literal_value(op.sep)
return (
pl.when(arg.list.len() > 0)
.then(arg.list.join(sep))
.otherwise(None)
)What version of ibis are you using?
10.1.0
What backend(s) are you using, if any?
polars
Relevant log output
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Type
Projects
Status
done