Delta Table already exists with `write_deltalake(..., mode='overwrite', overwrite_schema=True)`

[rspears74]

Environment

Delta-rs version: 0.15.1

Binding: Python

Environment:

• Cloud provider: AWS
• OS: MacOS
• Other: Github Action

---

Bug

What happened:
Calling write_deltalake(path, data, mode='overwrite', overwrite_schema=True, storage_options=...) fails with Delta Table already exists at that location.

What you expected to happen:
The existing Delta table to be overwritten.

How to reproduce it:
Call the above function to write to a path with an existing Delta table.

More details:

[rspears74]

Changing the path seems to have fixed this. But it seems the writer cannot handle a non-existent AWS S3 prefix. So if you have the following bucket structure:
Bucket: bucket

• path1
• path2
  And you try to write to s3://bucket/path1/new-table, it works.
  But if you try to write to s3://bucket/path1/new-location/new-table, it doesn't.

[ion-elgreco]

@rspears74 is this still the case with latest 0.22.3?

[ion-elgreco]

No response, so will conclude this is not relevant anymore