Open
Description
Please describe the issue
Hello!
First: thanks for this amazing project :)
I noted that databricks
is now enabled as a new backend (#10223, it's also listed under Backend support matrix), but I see no documentation on how to make the connection properly.
Digging into the source code, I noted
- the need for
DATABRICKS_SERVER_HOSTNAME
,DATABRICKS_HTTP_PATH
andDATABRICKS_TOKEN
env vars - the
.connect()
needs additional arguments:con = ibis.databricks.connect( catalog="main", schema="default", )
- it relies on Unity Catalog Volumes. At some point of the connection, it tries to create a Volume which is available only in the premium tiers (the need for Unity Catalog). When using a non-premium, I got:
ServerOperationError: [[UC_VOLUMES_NOT_ENABLED](https://docs.microsoft.com/azure/databricks/error-messages/error-classes#uc_volumes_not_enabled)] Support for Unity Catalog Volumes is not enabled on this instance. SQLSTATE: 56038 == SQL == CREATE VOLUME IF NOT EXISTS `root-py=31012-pid=2012` COMMENT 'Ibis memtable storage volume'
It would be awesome if those (and additional) minimal requirements were declared under the "Backends" section of the docs.
I've tested it using ibis-framework==10.2.0
, Python 3.10.
Thank you.
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Type
Projects
Status
backlog