List view

Is it possible to support PyArrow backed UDFs in Comet natively?

enhancementNew feature or requestNew feature or request

Status: Open.

#957 In apache/datafusion-comet;

· SemyonSinchenko opened on Sep 21, 2024

Register CometArrowAllocator as a Spark MemoryConsumer for JVM-UDF dispatch

area:expressionsExpression evaluationExpression evaluation

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4174 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Eliminate remaining row<->Arrow round-trip in CometPythonMapInArrowExec

area:udf

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4240 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Re-enable COUNT for mixed Spark partial / Comet final aggregate execution

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4242 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

[hdfs-opendal] Got an reader got too little data error on hdfs-opendal

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#2400 In apache/datafusion-comet;

· wForget opened on Sep 16, 2025

ExecutionMemoryPool errors releasing more memory than allocated

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2453 In apache/datafusion-comet;

· andygrove opened on Sep 25, 2025

Dropping a GlobalRef in a detached thread

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2470 In apache/datafusion-comet;

· andygrove opened on Sep 26, 2025

Enable windowed aggregates and fix correctness issues

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

correctness

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4007 In apache/datafusion-comet;

· andygrove opened on Apr 20, 2026

[native_datafusion] Add support for reading row index metadata columns

enhancementNew feature or requestNew feature or request

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#3432 In apache/datafusion-comet;

· andygrove opened on Feb 6, 2026

chore: DataFusion 54.0.0

buildBuild environmentBuild environment

dependenciesPull requests that update a dependency filePull requests that update a dependency file

enhancementNew feature or requestNew feature or request

Status: Open.

#3978 In apache/datafusion-comet;

· comphead opened on Apr 17, 2026

Implement JVM UDFs for all date/time expressions

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4311 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

native_datafusion more permissive than Spark 3.x when reading Parquet TimestampNTZ columns

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

spark 3.x

Status: Open.

#4219 In apache/datafusion-comet;

· andygrove opened on May 5, 2026

native_datafusion: ParquetSchemaConvert error does not include the file path

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#4316 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

FIRST/LAST returns wrong result with PartialMerge

bugSomething isn't workingSomething isn't working

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4131 In apache/datafusion-comet;

· comphead opened on Apr 28, 2026

Establish nomenclature style guide and audit operator names for clarity

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4419 In apache/datafusion-comet;

· andygrove opened on May 24, 2026

Add a docs-review Claude skill to enforce style guide and documentation quality

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4420 In apache/datafusion-comet;

· andygrove opened on May 24, 2026

Add distinction between "info" and "fallback" messages

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4006 In apache/datafusion-comet;

· andygrove opened on Apr 20, 2026

Math expression audit follow-ups (from #4486)

area:expressionsExpression evaluationExpression evaluation

correctness

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4500 In apache/datafusion-comet;

· andygrove opened on May 28, 2026

feat: support interval types and make_ym_interval / make_dt_interval

Status: Open (in progress).

apache/datafusion-cometnumber 4541

#4541 In apache/datafusion-comet;

· andygrove opened on May 30, 2026

deps: Upgrade to DataFusion 54.0.0

Status: Open (in progress).

apache/datafusion-cometnumber 4062

#4062 In apache/datafusion-comet;

· andygrove opened on Apr 24, 2026

docs: update CometBatchIterator references

documentationImprovements or additions to documentationImprovements or additions to documentation

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

Task

#4624 In apache/datafusion-comet;

· mbutrovich opened on Jun 11, 2026

[Bug] array_distinct / array_union / array_except do not canonicalize NaN like Spark

area:expressionsExpression evaluationExpression evaluation

correctness

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#4481 In apache/datafusion-comet;

· andygrove opened on May 28, 2026

[Bug] array_max and array_min disagree with Spark on NaN ordering

area:expressionsExpression evaluationExpression evaluation

correctness

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#4482 In apache/datafusion-comet;

· andygrove opened on May 28, 2026

feat: opt sort_array into codegen dispatch under strict floating-point mode

Status: Open (in progress).

apache/datafusion-cometnumber 4637

#4637 In apache/datafusion-comet;

· andygrove opened on Jun 12, 2026

feat: opt concat into codegen dispatch for non-UTF8_BINARY collations

Status: Open (in progress).

apache/datafusion-cometnumber 4640

#4640 In apache/datafusion-comet;

· andygrove opened on Jun 12, 2026