Skip to content

Conversation

@Fred1155
Copy link
Contributor

@Fred1155 Fred1155 commented Nov 4, 2025

Motivation and Context

According to public S3 documentation, https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-streaming.html , the chunk size is recomended to be at least 64 Kb to achieve better performance. The current default chunk size for sync is 128 KB and default chunk size for async is 16 Kb.

Modifications

Increase the DEFAULT_ASYNC_CHUNK_SIZE from 16Kb to 64Kb

Testing

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@Fred1155 Fred1155 requested a review from a team as a code owner November 4, 2025 00:15
@dagnir
Copy link
Contributor

dagnir commented Nov 4, 2025

The current default chunk size for sync is 128 KB

should we just increase to 128 KB to be consistent with sync?

* Default chunk size for Async trailer based checksum data transfer*
*/
public static final int DEFAULT_ASYNC_CHUNK_SIZE = 16 * 1024;
public static final int DEFAULT_ASYNC_CHUNK_SIZE = 64 * 1024;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only used by legacy code, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants