databricks workspace export-dir fails when directory contains an experiment #4081
Description
Describe the issue
databricks workspace export-dir fails when directory contains an experiment, which interrupts the entire export.
Basically, placing an experiment in a folder makes it impossible to export that folder (or workspace).
Steps to reproduce the behavior
- Put an MLflow experiment in a folder.
databricks workspace export-dir /my/folder .
Expected Behavior
Experiments are skipped, since they cannot be exported. Optionally this is done through a flag like --skip-non-exportable-files (not --ignore-errors, since that is not the same thing).
Actual Behavior
Error: No access to read file at /my/folder/some_experiment
CalledProcessError: Command 'databricks workspace export-dir /my/folder .' returned non-zero exit status 1.
OS and CLI version
Databricks CLI v0.278.0 on Linux.
Is this a regression?
I don't think so.
Debug Logs
databricks workspace export-dir /my/folder . --log-level=debug
17:42:53 Info: start pid=5465 version=0.278.0 args="databricks, workspace, export-dir, /my/folder, ., --log-level=debug"
Exporting files from /my/folder
17:42:53 Debug: GET /api/2.0/workspace/get-status?path=/my/folder&return_export_info=true
< HTTP/2.0 200 OK
< {
< "object_id": 854473616334895,
< "object_type": "DIRECTORY",
< "path": "/my/folder",
< "resource_id": "854473616334895"
< } pid=5465 sdk=true
17:42:53 Debug: GET /api/2.0/workspace/list?path=/my/folder
< HTTP/2.0 200 OK
< {
< "objects": [
< {
< "created_at": 1764280329361,
< "language": "PYTHON",
< "modified_at": 1764689708442,
< "object_id": 854473616334896,
< "object_type": "NOTEBOOK",
< "path": "/my/folder/scorer",
< "resource_id": "854473616334896"
< },
< {
< "created_at": 1764280358141,
< "language": "PYTHON",
< "modified_at": 1764716894811,
< "object_id": 854473616334897,
< "object_type": "NOTEBOOK",
< "path": "/my/folder/evaluate",
< "resource_id": "854473616334897"
< },
< {
< "created_at": 1764280682180,
< "language": "PYTHON",
< "modified_at": 1764715313552,
< "object_id": 854473616334898,
< "object_type": "NOTEBOOK",
< "path": "/my/folder/router",
< "resource_id": "854473616334898"
< },
< {
< "object_id": 1134869216940494,
< "object_type": "MLFLOW_EXPERIMENT",
< "path": "/my/folder/evaluation",
< "resource_id": "1134869216940494"
< },
< {
< "created_at": 1764355783248,
< "language": "PYTHON",
< "modified_at": 1764716285470,
< "object_id": 2034512528268430,
< "object_type": "NOTEBOOK",
< "path": "/my/folder/lib",
< "resource_id": "2034512528268430"
< }
< ]
< } pid=5465 sdk=true
17:42:53 Debug: GET /api/2.0/workspace/get-status?path=/my/folder/evaluate&return_export_info=true
< HTTP/2.0 200 OK
< {
< "created_at": 1764280358141,
< "language": "PYTHON",
< "modified_at": 1764716894811,
< "object_id": 854473616334897,
< "object_type": "NOTEBOOK",
< "path": "/my/folder/evaluate",
< "repos_export_format": "JUPYTER",
< "resource_id": "854473616334897"
< } pid=5465 sdk=true
17:42:54 Debug: GET /api/2.0/workspace/export?direct_download=true&path=/my/folder/evaluate
< HTTP/2.0 200 OK
< <Streaming response> pid=5465 sdk=true
/my/folder/evaluate -> evaluate.py
17:42:54 Debug: GET /api/2.0/workspace/get-status?path=/my/folder/evaluation&return_export_info=true
< HTTP/2.0 200 OK
< {
< "object_id": 1134869216940494,
< "object_type": "MLFLOW_EXPERIMENT",
< "path": "/my/folder/evaluation",
< "resource_id": "1134869216940494"
< } pid=5465 sdk=true
17:42:54 Debug: GET /api/2.0/workspace/export?direct_download=true&path=/my/folder/evaluation
< HTTP/2.0 400 Bad Request
< {
< "error_code": "BAD_REQUEST",
< "message": "No access to read file at /my/folder/evaluation"
< } pid=5465 sdk=true
17:42:54 Debug: non-retriable error: No access to read file at /my/folder/evaluation pid=5465 sdk=true
Error: No access to read file at /my/folder/evaluation
17:42:54 Info: failed execution pid=5465 exit_code=1 error="No access to read file at /my/folder/evaluation"
17:42:54 Debug: no telemetry logs to upload pid=5465