Summary
The default experimental.internal-communication.max-task-update-size is currently 16MB, which can be too small for large production queries with complex task updates.
Motivation
PR #27220 already had to override this property to 32MB for StandaloneQueryRunner after a generated wide-column test crossed the 16MB default (TaskUpdate size of 16.73MB has exceeded the limit of 16MB). We have also seen production query failures around the 30-40MB range, with edge cases around 100MB.
The older issue #14129 also shows that real task updates can exceed 16MB by a significant margin.
Proposal
Increase the default value of experimental.internal-communication.max-task-update-size from 16MB to 32MB. This aligns the product default with the OSS test override added in #27220 while keeping the limit conservative and still configurable for deployments that need a higher or lower value.
Summary
The default
experimental.internal-communication.max-task-update-sizeis currently16MB, which can be too small for large production queries with complex task updates.Motivation
PR #27220 already had to override this property to
32MBforStandaloneQueryRunnerafter a generated wide-column test crossed the16MBdefault (TaskUpdate size of 16.73MB has exceeded the limit of 16MB). We have also seen production query failures around the 30-40MB range, with edge cases around 100MB.The older issue #14129 also shows that real task updates can exceed 16MB by a significant margin.
Proposal
Increase the default value of
experimental.internal-communication.max-task-update-sizefrom16MBto32MB. This aligns the product default with the OSS test override added in #27220 while keeping the limit conservative and still configurable for deployments that need a higher or lower value.