Skip to content

bug: Databricks error when username contains a period #11950

@ShawInnes

Description

@ShawInnes

What happened?

Connecting with the following standard connection I would expect it to connect as normal.

If the local user id contains a period eg. first.last the Ibis memtable volume creation fails.

db = ibis.databricks.connect(
    server_hostname=databricks_host,
    http_path=databricks_warehouse,
    access_token=databricks_token,
    catalog="abc",
    schema="def",
)

The error: databricks.sql.exc.ServerOperationError: [UC_VOLUME_NOT_FOUND] Volume demo.reference.first.last-py=31210-pid=89423 does not exist. Please use 'SHOW VOLUMES' to list available volumes. SQLSTATE: 42704

Explicitly specifying a memtable volume parameter seems to get around this issue and the connection succesds

db = ibis.databricks.connect(
    server_hostname=databricks_host,
    http_path=databricks_warehouse,
    access_token=databricks_token,
    catalog="abc",
    schema="def",
    memtable_volume='ibis-volume'
)

What version of ibis are you using?

v12.0.0

What backend(s) are you using, if any?

Databricks

Relevant log output

Traceback (most recent call last):
  File "/var/folders/p_/wkhgr16x5459m6q_rbp7l6y00000gn/T/marimo_89423/__marimo__cell_MJUe_.py", line 1, in <module>
    db = ibis.databricks.connect(
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/ibis/__init__.py", line 110, in connect
    return backend.connect(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/ibis/backends/__init__.py", line 1052, in connect
    new_backend.reconnect()
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/ibis/backends/__init__.py", line 1067, in reconnect
    self.do_connect(*self._con_args, **self._con_kwargs)
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/ibis/backends/databricks/__init__.py", line 418, in do_connect
    self._post_connect(memtable_volume=memtable_volume)
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/ibis/backends/databricks/__init__.py", line 446, in _post_connect
    cur.execute(sql)
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/telemetry/latency_logger.py", line 182, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/client.py", line 1324, in execute
    self.active_result_set = self.backend.execute_command(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/backend/thrift_backend.py", line 1054, in execute_command
    execute_response, has_more_rows = self._handle_execute_response(
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/backend/thrift_backend.py", line 1261, in _handle_execute_response
    final_operation_state = self._wait_until_command_done(
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/backend/thrift_backend.py", line 940, in _wait_until_command_done
    self._check_command_not_in_error_or_closed_state(
  File "/Users/first.last/databricks/.venv/lib/python3.12/site-packages/databricks/sql/backend/thrift_backend.py", line 633, in _check_command_not_in_error_or_closed_state
    raise ServerOperationError(
databricks.sql.exc.ServerOperationError: [UC_VOLUME_NOT_FOUND] Volume `demo`.`reference`.`first`.`last-py=31210-pid=89423` does not exist. Please use 'SHOW VOLUMES' to list available volumes. SQLSTATE: 42704

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions