Skip to content

Add support for variable length strings #62

@dragly

Description

@dragly

Currently, structured NumPy arrays work just fine in our Python API, although they might not be supported when reading the data back in other languages. This means that data from for instance Pandas can be saved and loaded using Pandas.to_records().

However, we do not support variable length strings, because these appear as objects in the dtype, and hence become object arrays, which are not allowed (see #47) because they need to be pickled.

We should look into ways of storing variable length strings. However, these are not trivial to implement on top of the simple NumPy format, so we might need to consider adding a different backend for this purpose. My best bet for a cross-platform and lightweight format is SQLite, but that is still a large dependency to pull in for a single feature.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions