Coroutine allocation does not respect alignment when using the standard memory allocator

[Dugy]

I have found that although coroutines generally work with objects requiring special allocation, the specific combination of using Seastar's coroutines without the Seastar allocator causes segmentation faults. The segmentation faults happen because the objects appear in memory with wrong alignment, at addresses not divisible by their sizes. The issue is gone if I allocate these objects using a unique_ptr, but it's a performance eating workaround.

Steps to reproduce

1. Create a coroutine whose frame contains an object with unusual alignment requirements (such as std::experimental::fixed_size_simd<char, 32> from <experimental/simd> of GCC or Clang)
2. Run the coroutine without the Seastar allocator (diasbled by setting options.smp_opts.memory_allocator to seastar::memory_allocator::standard)
3. Ensure it's run a few times in a loop, since there's a chance the coroutine will coincidentally appear at a location with correct alignment
4. Compile in Release with debug symbols and run in a debugger

A small program to reproduce

There's a small chance it won't crash because the alignment will happen to be correct in all attempts.

#include <seastar/core/seastar.hh>
#include <seastar/core/reactor.hh>
#include <seastar/core/future-util.hh>
#include <seastar/net/api.hh>
#include <seastar/core/app-template.hh>
#include <seastar/core/reactor.hh>
#include <seastar/core/sleep.hh>
#include <string_view>
#include <experimental/simd>

seastar::future<char> doSimd() {
	std::experimental::fixed_size_simd<char, 32> breaker;
	constexpr std::string_view sourceData = "We just need some data that would be written to a structure with bad alignment and crash";
	breaker.copy_from(sourceData.data(), std::experimental::vector_aligned);
	co_await seastar::sleep(std::chrono::milliseconds(1));
	co_return breaker[0];
}

int main(int argc, char** argv)
{
	seastar::app_template::seastar_options options;
	options.smp_opts.memory_allocator = seastar::memory_allocator::standard;
	seastar::app_template app{std::move(options)};
	return app.run(argc, argv, [&] {
		std::vector<seastar::future<char>> futures;
		for (int i = 0; i < 4; i++) {
			futures.push_back(seastar::smp::submit_to(i, [] {
				return doSimd();
			}));
		}
		return seastar::when_all_succeed(futures.begin(), futures.end()).then([] (std::vector<char>) {
			std::cout << "Done.\n";
		});
	});
}

[crackcomm]

I tried to reproduce it on clang 22 but couldn't, what version of the compiler did you use?

[Dugy]

I was using GCC 15.

[crackcomm]

I am wondering if that would be enough:

diff --git a/include/seastar/core/coroutine.hh b/include/seastar/core/coroutine.hh
index 79d9777fc5..366459f82a 100644
--- a/include/seastar/core/coroutine.hh
+++ b/include/seastar/core/coroutine.hh
@@ -38,6 +38,7 @@
 extern "C" {
 
 void free_sized(void* ptr, size_t size);
+void free_aligned_sized(void* ptr, size_t alignment, size_t size);
 
 }
 
@@ -75,13 +76,27 @@
         memory::scoped_critical_alloc_section _;
         return ::malloc(size);
     }
+    static void* operator new(size_t size, std::align_val_t align) {
+        memory::scoped_critical_alloc_section _;
+        void* ptr = nullptr;
+        if (::posix_memalign(&ptr, static_cast<size_t>(align), size) != 0) {
+            throw std::bad_alloc();
+        }
+        return ptr;
+    }
     static void operator delete(void* ptr) noexcept {
         ::free(ptr);
     }
+    static void operator delete(void* ptr, std::align_val_t /*align*/) noexcept {
+        ::free(ptr);
+    }
 #ifdef __cpp_sized_deallocation
     static void operator delete(void* ptr, std::size_t sz) noexcept {
         ::free_sized(ptr, sz);
     }
+    static void operator delete(void* ptr, std::size_t sz, std::align_val_t align) noexcept {
+        ::free_aligned_sized(ptr, static_cast<size_t>(align), sz);
+    }
 #endif
 };

[Dugy]

No, it doesn't fix the problem. Debugger shows that it always calls the overload of operator new that only takes one argument and it ignores the other one.

[brain5lug]

Seems works as described in the cpp standart and even fix didn't make it and was abandoned:
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p2014r2.html
cplusplus/papers#750
https://eel.is/c++draft/dcl.fct.def.coroutine

Probably clang don't strictly follow the standard 😕