feat: Add Spark-dialect map_from_arrays to Velox

[emilysun201309]

Summary:
map_from_arrays is a Spark-only SQL name, but Velox bound it to the Presto
implementation class MapFunction<AllowDuplicateKeys=true>
(prestosql/Map.cpp). That variant skips the duplicate-key check entirely: it
neither throws nor dedupes, emitting a MapVector that physically contains both
entries for a repeated key. The result is not a valid map, and element_at
then reads whichever entry it hits first.

This adds MapFromArraysFunction in sparksql/Map.cpp, mirroring Spark's
ArrayBasedMapBuilder:

• length mismatch and null keys throw, with Spark's wording
• under EXCEPTION, a repeated key raises DUPLICATED_MAP_KEY
• under LAST_WIN, the repeated key's value overwrites the value of its FIRST
  occurrence in place, so the key keeps its first position.

Nulls follow Spark rather than Presto: only a top-level null key is rejected.
Presto additionally rejects an indeterminate key — a complex key containing a
null — which Spark accepts. BaseVector::equalValueAt compares with
kNullAsValue, so dedup treats two such keys as equal, consistent with Spark.

Deduplication orders keys by BaseVector::compare, for every key type rather
than only the ones that cannot be hashed. compare already reports NaN equal
to NaN and -0.0 equal to 0.0, which is the same equality dedup owes
equalValueAt; a hashed fast path has to reproduce that agreement by hand, and
keeping one strategy removes that obligation along with the type dispatch.

Three choices keep the common paths cheap:

• registration is stateful, so the dedup policy is resolved once when the
  expression is compiled rather than read from the query config on every batch.
  The previous per-apply() read was guarded on a possibly-null queryCtx and
  silently defaulted to LAST_WIN.
• under EXCEPTION no row can shrink, so when both arguments are identity-mapped
  and their arrays agree on offset and size, the result references the input
  arrays directly — no entry index buffers and no dictionary wrap.
• a constant key array is deduped once instead of per row; a row then writes
  only its value indices.

dedup logic and time complexity

key type	Spark	Presto	this diff
atomic, non-binary	HashMap, O(n)	sort, O(n log n)	ordered, O(n log n)
binary	TreeMap, O(n log n)	sort, O(n log n)	ordered, O(n log n)
complex	TreeMap, O(n log n)	sort, O(n log n)	ordered, O(n log n)

The null-key scan is skipped when the key elements vector reports no nulls, and
the fast-path predicate keeps the name, contract and asserts of
canTakeFastPath in prestosql/Map.cpp, so the two read alike. isNullAt is
virtual, so the guard removes a call per key on the common path.

Differential Revision: D115361136

[netlify]

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	bc751f8
🔍 Latest deploy log	https://app.netlify.com/projects/meta-velox/deploys/6a88d30502dd130008518f2d

[meta-codesync]

@emilysun201309 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D115361136.

[github-actions]

Selective Build Plan

Linux release with adapters is running a selective build of 20 cmake targets (out of 622 total). See the CI workflows README for what this means.

Affected targets (20)

Directly changed (3)

Target	Changed Files
velox_functions_spark	RegisterMap.cpp
velox_functions_spark_impl	Map.cpp
velox_functions_spark_test	MapFromArraysTest.cpp

Transitively affected (17)

• spark_expression_fuzzer_test
• velox_date_extract_benchmark
• velox_expression_runner
• velox_expression_runner_test
• velox_expression_runner_unit_test
• velox_functions_spark_aggregates_test
• velox_spark_function_registry_test
• velox_spark_query_runner_test
• velox_sparksql_benchmarks_cast
• velox_sparksql_benchmarks_compare
• velox_sparksql_benchmarks_from_json
• velox_sparksql_benchmarks_get_funcs
• velox_sparksql_benchmarks_hash
• velox_sparksql_benchmarks_in
• velox_sparksql_benchmarks_simd_compare
• velox_sparksql_benchmarks_split
• velox_sparksql_coverage

Unresolved (2)

These files could not be mapped to any target; a full build may be needed if they are build-relevant.

• velox/docs/functions/spark/map.rst
• velox/functions/sparksql/tests/CMakeLists.txt

---

Slow path • Graph generated from PR branch