Skip to content

Optimize memory usage for repartitioning when the distribution type is REPLICATED #14448

Open
@yingsu00

Description

@yingsu00

The PartitionedOutputOperator always populates each partition no matter if it's REPLICATED or PARTITIONED. This is not necessary when it's REPLICATED repartitioning. In such case, we only need to populate one partition so that the memory usage can be dramatically reduced.

Metadata

Metadata

Assignees

Labels

ariaPresto Aria performance improvementsstale

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions