Skip to content

add package deltalake integration#13

Draft
leo-schick wants to merge 2 commits intomainfrom
add-deltalake-integration
Draft

add package deltalake integration#13
leo-schick wants to merge 2 commits intomainfrom
add-deltalake-integration

Conversation

@leo-schick
Copy link
Member

@leo-schick leo-schick commented Feb 20, 2023

Adds simple integration support for package deltalake:

It adds a class mara_storage.integrations.deltalake.DeltaTable which creates a deltalake.DeltaTable but supports in addition the following parameters:

  • storage: the mara storage class or alias to be passed
  • path: The path inside the storage.

When storage is used, you can create the table simply with the mara alias. Example:

from mara_storage.integrations.deltalake import DeltaTable

table = DeltaTable('sales/invoices', storage='lakehouse')  # where 'lakehouse' is your storage alias

# returns the parquet files in the delta table of the current version
print(table.files())

Open tasks

@leo-schick leo-schick force-pushed the add-deltalake-integration branch from 14b0b98 to 99ec58a Compare February 22, 2023 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant