[dagster-polars] Feat/delta merge options

[boccileonardo]

Summary & Motivation

I would recommend a new minor rather than patch, since this adds a significant bit of functionality.

• Add support for delta merge operation.

How I Tested These Changes

• Added unit tests

Changelog

Ensure that an entry has been created in CHANGELOG.md outlining additions, deletions, and/or modifications.

See: keepachangelog.com

[boccileonardo]

@danielgafni I would need a bit of help with the open TODOs, to decide if the current approach is fine or not for missing target tables while merging and with order of operations in case of multiple matching condition.

But most of all I need some help with the handling of multiple partitions, to add the correct conditions to the merge condition and allow partition pruning and avoid a full table scan.
Basically, could you point me to some docs or example of what I would find in here in a multi-partitioned asset?

if isinstance(partition_by, dict) and isinstance(
    context.partition_key, MultiPartitionKey
):
    delta_write_options["partition_by"] = list(partition_by.values())

[boccileonardo]

Hey @danielgafni, did you have a chance to take a look? I'm fine with leaving the TODOs as they are right now, but I'd love some help on the multi-partition handling as I'm not too familiar with it.

[danielgafni]

Hey, not yet, I was busy at work. I'll try to take a look today or tomorrow!