Support APPEND for insert_existing_partitions_behavior and align default with immutable_partitions

[wongquijote]

In Java Presto, HiveClientConfig.getInsertExistingPartitionsBehavior() couples the default to immutablePartitions: it returns APPEND when partitions are mutable, and ERROR/OVERWRITE when they are immutable.

Velox C++ diverges in two related ways:

1. APPEND is not a recognized behavior at all for partitioned tables.
   - InsertExistingPartitionsBehavior enum has only kError and kOverwrite (HiveConfig.cpp:30,33).
   - stringToInsertExistingPartitionsBehavior only parses "ERROR" and "OVERWRITE".
   - The switch in HiveDataSink::getUpdateMode() (HiveDataSink.cpp:837) has no kAppend case — the default: arm throws VELOX_UNSUPPORTED.
2. The catalog-level default is hardcoded to "ERROR" (HiveConfig.cpp:87), independent of immutablePartitions.

Interestingly, the unpartitioned existing-table path already implements the Java-style behavior: returns kAppend when mutable, fails when immutable (HiveDataSink.cpp:854-857). So immutablePartitions is consulted there — just not for the partitioned-table path, where it's never read.

I'd like to support APPEND for existing partitions. Before I make changes, I wanted to surface this asymmetry and check whether it was intentional. My proposed direction:

• Add kAppend to the enum, the string parser, and the getUpdateMode() switch.
• Decide whether the catalog default should mirror Java (coupled to immutablePartitions) or remain independent and explicit.

[wongquijote]

cc @aditi-pandit

[aditi-pandit]

@mbasmanova @xiaoxmeng : Wanted to bring your attention to this issue.

Seems like Velox honors HiveConfig for immutablePartitions for un-partitioned tables while it ignores it for partitioned tables. Its odd that we consider immutablePartitions for unpartitioned tables (which shouldn't have partitions) vs partitioned tables that do.

velox/velox/connectors/hive/HiveDataSink.cpp

Line 837 in 7e73421

WriterParameters::UpdateMode HiveDataSink::getUpdateMode() const {

We want to add Append behavior for partitioned tables and use immutablePartitions config for parity with Presto Java if we can.

Is there any reason for ignoring immutablePartitions config when inserting into partitioned tables ? I would imagine this config is not set in your clusters then.

@amitkdutta

[apurva-meta]

Good catch @wongquijote. On the config question -> immutablePartitions does matter in practice; it's just that in our setup partitioned inserts always resolve to ERROR or OVERWRITE, never APPEND (we run with immutable partitions, and use OVERWRITE explicitly where we need it). Those happen to be the two values the native path already supports, which is why the missing kAppend hasn't surfaced for us even though Velox ignores immutablePartitions for the partitioned case. It reads more like an incomplete port than a deliberate decision — the unpartitioned path checks immutablePartitions, the partitioned one doesn't.

+1 on adding kAppend (enum + string parser + getUpdateMode() switch) —> that part is purely additive.

One thing to be careful about is the default. In Prestissimo the worker computes getUpdateMode() from its own config — the coordinator doesn't bake the mode into the serialized handle, and the coordinator-side immutablePartitions value doesn't necessarily propagate to the worker. So a worker that doesn't explicitly set immutable-partitions will see false. If the partitioned default is derived from immutablePartitions to "mirror Java," it'd resolve to APPEND there instead of ERROR — silently turning insert-into-existing-partition from fail-fast into append, which is the opposite of Java and a data-duplication risk. The "reject APPEND when immutable" guard wouldn't catch it, since immutable is already false on the worker.

On rollout — I don't think kAppend needs a dedicated feature flag. As long as the default stays ERROR, adding it is inert: it only activates when someone explicitly sets insert-existing-partitions-behavior=APPEND (per-session or per-catalog), so existing deployments are unaffected and append can be enabled one catalog at a time. The config value is effectively the gate. So I'd keep the default decoupled from immutablePartitions (default ERROR) and treat APPEND as strictly opt-in.

Last thing worth preserving: Java still blocks append into an existing partition of a bucketed table, so the kAppend path should keep that restriction.