Feature Request: lance table format

[Borg93]

Support other format liks lance? Why only parquet?

[ukclivecox]

Or Vortex. Be good to know pros/cons - viability.

[hpvd]

there was just the release of lance 2.1 to stable with interesting refinements:
https://lancedb.com/blog/lance-file-2-1-stable/

[hpvd]

see also: #204

[hpvd]

btw: there is a lance extension for duckdb
https://github.com/lancedb/lance-duckdb

[hpvd]

just fyi:
duckDBs lance extension makes stunning progress and is now ready to go:
https://github.com/lance-format/lance-duckdb/releases

and it's directly listed as duckdb community extension:
https://duckdb.org/community_extensions/extensions/lance

[hpvd]

lance DuckDB extension is now fairly mature. There is a new blog post describing the current state and the possibilities:
https://lancedb.com/blog/lance-x-duckdb-sql-retrieval-on-the-multimodal-lakehouse-format/

[hpvd]

just for everyone interested:

lance is now a core extension of duckdb:
https://duckdb.org/docs/current/core_extensions/lance

and there are some new supported features in latest versions..
https://github.com/lance-format/lance-duckdb/releases

[SamuelLHuber]

currently from the duckdb spec:

Data storage: The DuckLake specification requires a storage component for storing the data in Parquet format.

so it seems one has to currently replace parquet and implement an adjusted version of the duckdb spec for using Lance.

Otherwise a spec update seems required to introduce other data formats like lance.