Use std::align_alloc in file_data_loader #10660
Open
+110
−62
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10660
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit f4391a7 with merge base be2dda7 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
changed the title
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 2, 2025
Summary: Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Differential Revision: D74041198
larryliu0820
approved these changes
May 2, 2025
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 2, 2025
Summary: Pull Request resolved: pytorch#10660 Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Reviewed By: larryliu0820 Differential Revision: D74041198
alignment is covered by the file_data_loader_tests: I think we shouldn't have this error now that we've moved to aligned_alloc (hopefully), probably the main thing is to make sure oss ci passes on macos. Are you thinking of a different test though? |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 2, 2025
Summary: Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Reviewed By: larryliu0820 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 3, 2025
Summary: Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 3, 2025
Summary: Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 3, 2025
Summary: Pull Request resolved: pytorch#10660 Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 3, 2025
Summary: |... Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. For systems that do not have aligned_alloc (or posix_memalign) fallback to malloc. The malloc implementation is similar to what file_data_loader.cpp does. Except, we do not have a custom free function from FreeableBuffer, so we store an offset just before the aligned ptr to free the actual buffer. 1. Allocate via malloc; buffer = malloc(size + sizeof(uint16_t) + alignment - 1) - size: the size requested. - sizeof(uint16_t): a place to store the offset between the aligned buffer and the original ptr. - alignment-1: extra padding to allow for alignment. 2. Align (buffer + sizeof(uint16_t)) to alignment. This (usually) pushes the buffer forward by `alignment`. 3. Store the difference of (aligned_ptr - buffer). The memory will look like this: | buffer start | maybe padding | offset (aligned_buffer - buffer) | aligned_buffer start | maybe padding | buffer end | We should have between offset_size (2) and 1 aligned block before the actual aligned buffer. https://embeddedartistry.com/blog/2017/02/22/generating-aligned-memory/ Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 5, 2025
Summary: |... Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. For systems that do not have aligned_alloc (or posix_memalign) fallback to malloc. The malloc implementation is similar to what file_data_loader.cpp does. Except, we do not have a custom free function from FreeableBuffer, so we store an offset just before the aligned ptr to free the actual buffer. 1. Allocate via malloc; buffer = malloc(size + sizeof(uint16_t) + alignment - 1) - size: the size requested. - sizeof(uint16_t): a place to store the offset between the aligned buffer and the original ptr. - alignment-1: extra padding to allow for alignment. 2. Align (buffer + sizeof(uint16_t)) to alignment. This (usually) pushes the buffer forward by `alignment`. 3. Store the difference of (aligned_ptr - buffer). The memory will look like this: | buffer start | maybe padding | offset (aligned_buffer - buffer) | aligned_buffer start | maybe padding | buffer end | We should have between offset_size (2) and 1 aligned block before the actual aligned buffer. https://embeddedartistry.com/blog/2017/02/22/generating-aligned-memory/ Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 5, 2025
Summary: |... Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. For systems that do not have aligned_alloc (or posix_memalign) fallback to malloc. The malloc implementation is similar to what file_data_loader.cpp does. Except, we do not have a custom free function from FreeableBuffer, so we store an offset just before the aligned ptr to free the actual buffer. 1. Allocate via malloc; buffer = malloc(size + sizeof(uint16_t) + alignment - 1) - size: the size requested. - sizeof(uint16_t): a place to store the offset between the aligned buffer and the original ptr. - alignment-1: extra padding to allow for alignment. 2. Align (buffer + sizeof(uint16_t)) to alignment. This (usually) pushes the buffer forward by `alignment`. 3. Store the difference of (aligned_ptr - buffer). The memory will look like this: | buffer start | maybe padding | offset (aligned_buffer - buffer) | aligned_buffer start | maybe padding | buffer end | We should have between offset_size (2) and 1 aligned block before the actual aligned buffer. https://embeddedartistry.com/blog/2017/02/22/generating-aligned-memory/ Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
lucylq
added a commit
to lucylq/executorch-1
that referenced
this pull request
May 5, 2025
Summary: |... Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. For systems that do not have aligned_alloc (or posix_memalign) fallback to malloc. The malloc implementation is similar to what file_data_loader.cpp does. Except, we do not have a custom free function from FreeableBuffer, so we store an offset just before the aligned ptr to free the actual buffer. 1. Allocate via malloc; buffer = malloc(size + sizeof(uint16_t) + alignment - 1) - size: the size requested. - sizeof(uint16_t): a place to store the offset between the aligned buffer and the original ptr. - alignment-1: extra padding to allow for alignment. 2. Align (buffer + sizeof(uint16_t)) to alignment. This (usually) pushes the buffer forward by `alignment`. 3. Store the difference of (aligned_ptr - buffer). The memory will look like this: | buffer start | maybe padding | offset (aligned_buffer - buffer) | aligned_buffer start | maybe padding | buffer end | We should have between offset_size (2) and 1 aligned block before the actual aligned buffer. https://embeddedartistry.com/blog/2017/02/22/generating-aligned-memory/ Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
Summary: Issue with aligned buffers: P1800967583 The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment. However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer, we overflow and error out. Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4) ``` The malloc(), calloc(), realloc(), and reallocarray() functions return a pointer to the allocated memory, which is suitably aligned for any type that fits into the requested size or less. ``` Use std::aligned_alloc (C++17) to ensure buffer is aligned. For systems that do not have aligned_alloc (or posix_memalign) fallback to malloc. The malloc implementation is similar to what file_data_loader.cpp does. Except, we do not have a custom free function from FreeableBuffer, so we store an offset just before the aligned ptr to free the actual buffer. 1. Allocate via malloc; buffer = malloc(size + sizeof(uint16_t) + alignment - 1) - size: the size requested. - sizeof(uint16_t): a place to store the offset between the aligned buffer and the original ptr. - alignment-1: extra padding to allow for alignment. 2. Align (buffer + sizeof(uint16_t)) to alignment. This (usually) pushes the buffer forward by `alignment`. 3. Store the difference of (aligned_ptr - buffer). The memory will look like this: | buffer start | maybe padding | offset (aligned_buffer - buffer) | aligned_buffer start | maybe padding | buffer end | We should have between offset_size (2) and 1 aligned block before the actual aligned buffer. https://embeddedartistry.com/blog/2017/02/22/generating-aligned-memory/ NOTE: this increase binary size (linux, clang) by 8 bytes, so also raising it there. Reviewed By: larryliu0820, mcr229 Differential Revision: D74041198
|
This pull request was exported from Phabricator. Differential Revision: D74041198 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Issue with aligned buffers: P1800967583
The alignment requested is 16, and std::max_align_t is also 16. This means we do not need to pad the size to meet any alignment.
However, the buffer we get from malloc is aligned to 8, not 16. When we try to align the buffer to 16, we overflow the original buffer (as it wasn't padded) and error out.
Seems like malloc is not guaranteed to return 8 or 16 byte-aligned buffers, so also a bit hard to test definitively. So far we've only seen this when the buffer size is small (size 2, 4)
Use std::aligned_alloc (C++17) to ensure buffer is aligned.
Differential Revision: D74041198