Provide a way to rebuild the sysroot with specific compiler flags

[valadaptive]

This is a follow-up to #547. I've determined that the difference in code size is down to libc and libcxx being compiled using -O3 instead of -Oz.

Emscripten rebuilds and caches the sysroot libraries based on the compiler flags you provide, so if you compile your own code using -Oz, Emscripten will too. wasi-sdk, on the other hand, builds the toolchain and the sysroot as part of the same process and ships precompiled libraries. It's not straightforward to build the sysroot using a prebuilt toolchain.

Building wasi-libc using custom settings isn't too difficult, although pointing future builds to use the custom-built wasi-libc is a bit annoying. It's pretty easy for a WebAssembly project to include wasi-libc as a Git submodule and build it as part of their own build process.

Building libcxx is more involved, since it's part of LLVM. The CMake configurations and such are part of this repo. Building libcxx requires:

• Cloning this repo, including all of llvm-project. Even a shallow clone takes a while because LLVM is huge.
• Making sure you check out the repo version corresponding to the released wasi-sdk version you'll use to build it.
• Using the prebuilt wasi-sdk toolchain to build the wasi-sdk sysroot using your desired settings.

As such, I'm wondering if it would make sense to release source distributions of wasi-libc, libcxx, libcxxabi, and compiler-rt (plus wasi-sdk-sysroot.cmake). This would allow for easily rebuilding the full sysroot without cloning the entire LLVM repo.

[alexcrichton]

It's not straightforward to build the sysroot using a prebuilt toolchain.

FWIW locally I ran:

cmake -B build -S . -DCMAKE_C_COMPILER=clang-19 -DCMAKE_CXX_COMPILER=clang++-19 -G Ninja -DWASI_SDK_CPU_CFLAGS= -DWASI_SDK_TARGETS=wasm32-wasip2

Afterwards I could use:

clang++-19 foo.cpp --target=wasm32-wasip2 --sysroot=./build/install/share/wasi-sysroot -resource-dir=./build/install/wasi-resource-dir

to build and compile a file. Documentation around this isn't great, but it's intended to be at least not a herculean task to build everything. Building with custom flags is done through -DCMAKE_C_FLAGS and friends.

I'm wondering if it would make sense to release source distributions of wasi-libc, libcxx, libcxxabi, and compiler-rt

Disentangling the projects from LLVM would require disentangling their build systems, which is not really something we're ready to do. More-or-less that's why precompiled builds are provided here, downloading/building LLVM is not easy. If you know of a way to disentangle the build systems yourself, this'd be an interesting PR!

---

I'll note that unlike Emscripten I think it's pretty unlikely wasi-sdk will lazily recompile the sysroot based on the flags you pass. If you want a custom sysroot it's intended that you can compile your own as opposed to having wasi-sdk manage it for you

[ncruces]

Something like this might also make it easier to use the (optional) SIMD libc primitives I contributed:
https://github.com/WebAssembly/wasi-libc/pulls?q=optional+simd

Which, I imagine, see very little use as is.

[sbc100]

Emscripten rebuilds and caches the sysroot libraries based on the compiler flags you provide, so if you compile your own code using -Oz, Emscripten will too

This is not true. In emscripten we always compile system libraries with a specific set of flags. For example you can see we always use -Oz for libcxx: https://github.com/emscripten-core/emscripten/blob/7a5d93b50f6a3a35e85a0d2fc9e667b8498e6aed/tools/system_libs.py#L1693-L1705

So, while libraries can be compiled on demand, there do not come in an infinite variety.

[kripken]

Just to add to @sbc100 's point, there is a little nuance in Emscripten around -Oz specifically:

https://github.com/emscripten-core/emscripten/blob/7a5d93b50f6a3a35e85a0d2fc9e667b8498e6aed/tools/system_libs.py#L1404-L1407

That might be what you were seeing @valadaptive - in -Oz, slightly different system libs are built (focused around smaller size at all costs).