Skip to content

Conversation

sophieil
Copy link

@sophieil sophieil commented May 5, 2025

Fixes #8762

Also addresses the point about full object checksums mentioned in #8760, enabling full object checksums where applicable and adding support for CRC64NVME checksums.

Multipart checksum behaviour has been based on AWS docs here, verified using the attached script
moto-8762-aws-behaviour-demo.py.txt which produces the following output:

CRC64NVME - FULL_OBJECT: Object stored with FULL_OBJECT checksum.  Object Attributes response: 'SXtKxXGlB1A='.  Get Object response: 'SXtKxXGlB1A='.
CRC64NVME - COMPOSITE: not a supported combination
CRC64NVME - DEFAULT: Object stored with FULL_OBJECT checksum.  Object Attributes response: 'SXtKxXGlB1A='.  Get Object response: 'SXtKxXGlB1A='.

CRC32 - FULL_OBJECT: Object stored with FULL_OBJECT checksum.  Object Attributes response: '7PweJw=='.  Get Object response: '7PweJw=='.
CRC32 - COMPOSITE: Object stored with COMPOSITE checksum.  Object Attributes response: 'pd+GUQ=='.  Get Object response: 'pd+GUQ==-2'.
CRC32 - DEFAULT: Object stored with COMPOSITE checksum.  Object Attributes response: 'pd+GUQ=='.  Get Object response: 'pd+GUQ==-2'.

CRC32C - FULL_OBJECT: Object stored with FULL_OBJECT checksum.  Object Attributes response: 'yuafwA=='.  Get Object response: 'yuafwA=='.
CRC32C - COMPOSITE: Object stored with COMPOSITE checksum.  Object Attributes response: 'MIeOGQ=='.  Get Object response: 'MIeOGQ==-2'.
CRC32C - DEFAULT: Object stored with COMPOSITE checksum.  Object Attributes response: 'MIeOGQ=='.  Get Object response: 'MIeOGQ==-2'.

SHA1 - FULL_OBJECT: not a supported combination
SHA1 - COMPOSITE: Object stored with COMPOSITE checksum.  Object Attributes response: 'II0B/WXYv6uiHlCAB8ELEj2JIjo='.  Get Object response: 'II0B/WXYv6uiHlCAB8ELEj2JIjo=-2'.
SHA1 - DEFAULT: Object stored with COMPOSITE checksum.  Object Attributes response: 'II0B/WXYv6uiHlCAB8ELEj2JIjo='.  Get Object response: 'II0B/WXYv6uiHlCAB8ELEj2JIjo=-2'.

SHA256 - FULL_OBJECT: not a supported combination
SHA256 - COMPOSITE: Object stored with COMPOSITE checksum.  Object Attributes response: '1kJPjs+5gw2wOP9NXohhWkJdwr6FIR0Zsz3JFkaXgmk='.  Get Object response: '1kJPjs+5gw2wOP9NXohhWkJdwr6FIR0Zsz3JFkaXgmk=-2'.
SHA256 - DEFAULT: Object stored with COMPOSITE checksum.  Object Attributes response: '1kJPjs+5gw2wOP9NXohhWkJdwr6FIR0Zsz3JFkaXgmk='.  Get Object response: '1kJPjs+5gw2wOP9NXohhWkJdwr6FIR0Zsz3JFkaXgmk=-2'.

@sophieil sophieil marked this pull request as ready for review May 5, 2025 10:54
@sophieil
Copy link
Author

sophieil commented May 6, 2025

Thanks for triggering the checks @bblommers, I will look through and fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multipart S3 uploads - composite checksums are not given the -X suffix

1 participant