Skip to content

[feature](be) Add adaptive batch size for pipeline operators#62975

Open
mrhhsg wants to merge 2 commits intoapache:masterfrom
mrhhsg:abs_operators
Open

[feature](be) Add adaptive batch size for pipeline operators#62975
mrhhsg wants to merge 2 commits intoapache:masterfrom
mrhhsg:abs_operators

Conversation

@mrhhsg
Copy link
Copy Markdown
Member

@mrhhsg mrhhsg commented Apr 30, 2026

What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Extend adaptive batch size from the scan path to the remaining pipeline operators, including join, aggregation, exchange, union, table function, and sort outputs.

Release note

None

Check List (For Author)

  • Test: Unit Test
    • Unit Test: ./run-be-ut.sh --run --filter='AggOperatorTestWithOutGroupBy.:AggOperatorTestWithGroupBy.:DistinctStreamingAggOperatorTest.:ExchangeSourceOperatorXTest.:HashJoinProbeOperatorTest.:IntersectOperatorTest.:ExceptOperatorTest.:StreamingAggOperatorTest.:TableFunctionOperatorTest.:UnnestTest.:UnionOperatorTest.:FullSorterTest.:PartitionSorterTest.:SortMergerTest.:MergeSorterStateTest.*'
  • Behavior changed: Yes (adaptive batch sizing now applies to more pipeline operators)
  • Does this need documentation: No

mrhhsg added 2 commits April 30, 2026 15:07
Issue Number: None

Related PR: None

Problem Summary: Remove unused reader context and generic reader fields in the current staged changes so the reader path stays aligned with the current output-column and batch-size handling.

None

- Test: No need to test (commit current staged tracked changes only)
- Behavior changed: No
- Does this need documentation: No
### What problem does this PR solve?

Issue Number: None

Related PR: None

Problem Summary: Extend adaptive batch size from the scan path to the remaining pipeline operators, including join, aggregation, exchange, union, table function, and sort outputs.

### Release note

None

### Check List (For Author)

- Test: Unit Test
    - Unit Test: ./run-be-ut.sh --run --filter='AggOperatorTestWithOutGroupBy.*:AggOperatorTestWithGroupBy.*:DistinctStreamingAggOperatorTest.*:ExchangeSourceOperatorXTest.*:HashJoinProbeOperatorTest.*:IntersectOperatorTest.*:ExceptOperatorTest.*:StreamingAggOperatorTest.*:TableFunctionOperatorTest.*:UnnestTest.*:UnionOperatorTest.*:FullSorterTest.*:PartitionSorterTest.*:SortMergerTest.*:MergeSorterStateTest.*'
- Behavior changed: Yes (adaptive batch sizing now applies to more pipeline operators)
- Does this need documentation: No
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yiguolei
Copy link
Copy Markdown
Contributor

yiguolei commented May 1, 2026

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants