allow_unsafe_rename option stopped working in version 1

[djouallah]

Environment

Delta-rs version: 1

Binding:

Environment:

• Cloud provider:Fabric
• OS: Linux
• Other:

---

Bug

What happened:

getting error when running this

from  deltalake import write_deltalake
write_deltalake('/lakehouse/default/Tables/aemo/ca' , df, mode="overwrite", storage_options={"allow_unsafe_rename":"true"})

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In[7], line 2
      1 from  deltalake import write_deltalake
----> 2 write_deltalake('/lakehouse/default/Tables/aemo/ca' , df, mode="overwrite", storage_options={"allow_unsafe_rename":"true"})

File ~/jupyter-env/python3.11/lib/python3.11/site-packages/deltalake/writer/writer.py:142, in write_deltalake(table_or_uri, data, partition_by, mode, name, description, configuration, schema_mode, storage_options, predicate, target_file_size, writer_properties, post_commithook_properties, commit_properties)
    126     table._table.write(
    127         data=data,
    128         batch_schema=compatible_delta_schema,
   (...)
    139         post_commithook_properties=post_commithook_properties,
    140     )
    141 else:
--> 142     write_deltalake_rust(
    143         table_uri=table_uri,
    144         data=data,
    145         batch_schema=compatible_delta_schema,
    146         partition_by=partition_by,
    147         mode=mode,
    148         schema_mode=schema_mode,
    149         predicate=predicate,
    150         target_file_size=target_file_size,
    151         name=name,
    152         description=description,
    153         configuration=configuration,
    154         storage_options=storage_options,
    155         writer_properties=writer_properties,
    156         commit_properties=commit_properties,
    157         post_commithook_properties=post_commithook_properties,
    158     )

OSError: Generic LocalFileSystem error
          ↳ Unable to rename file
           ↳ No such file or directory (os error 2)

What you expected to happen:

it works fine with previous version when you pick engine as rust

How to reproduce it:

More details:

[ion-elgreco]

Strange, I can't reproduce this, I added some dbgs through the code to see if it got properly initiated, which it does.

export RUST_LOG="DEBUG" && pytest tests/test_mount.py::test_write_mount
================================================================================== test session starts ===================================================================================
platform darwin -- Python 3.13.2, pytest-8.3.4, pluggy-1.5.0 -- /Users/ion/Development/delta-rs/python/.venv/bin/python
cachedir: .pytest_cache
benchmark: 5.1.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/ion/Development/delta-rs/python
configfile: pyproject.toml
plugins: cov-6.0.0, benchmark-5.1.0, timeout-2.3.1, mock-3.14.0
collected 1 item

tests/test_mount.py::test_write_mount FAILED                                                                                                                                       [100%]

======================================================================================== FAILURES ========================================================================================
____________________________________________________________________________________ test_write_mount ____________________________________________________________________________________

tmp_path = PosixPath('/private/var/folders/r7/nkl64k_s60d234dltpvztj1r0000gn/T/pytest-of-ion/pytest-590/test_write_mount0')
sample_table = arro3.core.Table
+------+-------+-------+---------+
| id   | price | sold  | deleted |
| Utf8 | Int64 | Int32 | Boolea...   | false   |
| 4    | 3     | 3     | false   |
| 5    | 4     | 4     | false   |
+------+-------+-------+---------+

    def test_write_mount(tmp_path, sample_table: Table):
        write_deltalake(
            tmp_path,
            sample_table,
            mode="append",
            storage_options={"allow_unsafe_rename": "true"},
        )
>       assert False
E       assert False

tests/test_mount.py:13: AssertionError
---------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------
[2025-05-29T06:18:10Z DEBUG deltalake_core::table::builder] creating table builder with file:///private/var/folders/r7/nkl64k_s60d234dltpvztj1r0000gn/T/pytest-of-ion/pytest-590/test_write_mount0
[2025-05-29T06:18:10Z DEBUG deltalake_core::table::builder] build_storage() with file:///private/var/folders/r7/nkl64k_s60d234dltpvztj1r0000gn/T/pytest-of-ion/pytest-590/test_write_mount0
[crates/core/src/table/builder.rs:263:9] &storage_config = StorageConfig {
    runtime: None,
    retry: RetryConfig {
        backoff: BackoffConfig {
            init_backoff: 100ms,
            max_backoff: 15s,
            base: 2.0,
        },
        max_retries: 10,
        retry_timeout: 180s,
    },
    limit: None,
    unknown_properties: {
        "allow_unsafe_rename": "true",
    },
    raw: {
        "allow_unsafe_rename": "true",
    },
}
[2025-05-29T06:18:10Z DEBUG deltalake_core::table::builder] Loading a logstore based off the location: Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/private/var/folders/r7/nkl64k_s60d234dltpvztj1r0000gn/T/pytest-of-ion/pytest-590/test_write_mount0", query: None, fragment: None }
[crates/core/src/logstore/mod.rs:182:5] &scheme = Url {
    scheme: "file",
    cannot_be_a_base: false,
    username: "",
    password: None,
    host: None,
    port: None,
    path: "/",
    query: None,
    fragment: None,
}
[2025-05-29T06:18:10Z DEBUG deltalake_core::logstore] Found a storage provider for file:/// (file:///private/var/folders/r7/nkl64k_s60d234dltpvztj1r0000gn/T/pytest-of-ion/pytest-590/test_write_mount0)
[crates/mount/src/lib.rs:49:9] &options = {
    "allow_unsafe_rename": "true",
}
[crates/mount/src/lib.rs:52:9] &config = {
    AllowUnsafeRename: "true",
}
[crates/mount/src/lib.rs:60:9] &allow_unsafe_rename = true
[crates/mount/src/lib.rs:74:17] "imhere" = "imhere"
[crates/mount/src/lib.rs:76:21] "creating mount store" = "creating mount store"

[ion-elgreco]

No such file or directory indicates that either the path that's resolved to is incorrect or it got deleted before it had the chance to be renamed.

@djouallah can Microsoft give us access to fabric workspace for free, so we can test ourselves?

[djouallah]

@ion-elgreco thanks, let me ask around !!

[skriszo]

@ion-elgreco
@djouallah may I ask, what was the solution? (as you closed this ticket)

I am having exactly the same issue in Synapse notebook with mounted directory. For me it also stopped working after upgrading deltalake to above 1