Skip to content

bug: issues with Map(K,V) Clickhouse type #11795

@danielgafni

Description

@danielgafni

What happened?

Hi!

I've encountered two issues with Map(String, String):

  1. At write time, Ibis doesn't seem to convert Pyarrow struct types (or a Polars struct type) into the appropriate list[tuple[name, value]] representation. clickhouse-connect then fails to insert the dictionary. I guess Ibis should handle this? This conversion is trivial.

  2. At read time, Ibis doesn't do the opposite conversion, but this is also a harder one: Map keys are arbitrary while Arrow/Polars Struct has a know set of keys (struct schema).

I guess this is an edge case because this conversion could only work with K=String. Not sure if it's worth handling, mostly wanted to document it.

What version of ibis are you using?

11.0.0

What backend(s) are you using, if any?

ClickHouse

Relevant log output

clickhouse_connect.driver.exceptions.DatabaseError: Received ClickHouse exception, code: 53, server response: Code: 53. DB::Exception: CAST AS Map can only be performed from tuples of array. Got: Tuple(frames Nullable(String), audio Nullable(String)): while converting source column metaxy_provenance_by_field to destination column metaxy_provenance_by_field. (TYPE_MISMATCH) (version 25.11.2.24 (official build)) (for url http://localhost:8123)

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions