Merge upstream wasi-sdk (through wasi-sdk-33): C++ exceptions, libunwind, wasip2/p3#239
Draft
lewing wants to merge 179 commits into
Draft
Merge upstream wasi-sdk (through wasi-sdk-33): C++ exceptions, libunwind, wasip2/p3#239lewing wants to merge 179 commits into
lewing wants to merge 179 commits into
Conversation
Previously, the release process for wasi-sdk was undocumented and manual. The `RELEASING.md` document in this commit describes the steps necessary to publish a new version of `wasi-sdk` as a GitHub release and provides helpful scripts to automate some of the steps. With this in place, a future change could add a workflow trigger that allows running the steps automatically in GitHub actions. Keeping the steps as scripts in the repository, however, allows developers to re-run steps themselves in the (likely) case that something goes awry.
) Exclude dist-ubuntu-latest to prefer dist-ubuntu-bionic, which is compatible with wider distributions. cf. WebAssembly#273 (comment) WebAssembly#303
This change switches the MacOS build to target 10.12, an older version than currently supported. This would close dotnet#127. @sbc100 mentions that this is the setting used by Emscripten and related projects.
This change brings in several helpful PRs (e.g., WebAssembly/wasi-libc#397, WebAssembly/wasi-libc#399, WebAssembly/wasi-libc#401) in anticipation of creating a release based on LLVM 16.
* llvm-project: update to 16.0.0 release This changes updates the `src/llvm-project` submodule to the `HEAD` of `release/16.x`, the same commit used to [release] the LLVM 16.0.0 binaries. [release]: https://github.com/llvm/llvm-project/releases/tag/llvmorg-16.0.0 * fix: use only Clang's major version in install prefix Due to [a change] in LLVM, Clang will expect to find the `libclang_rt.builtins-wasm32.a` file in a path that only contains the major version (`16`) instead of the entire version (`16.0.0`) as was previously the case. This change modifies the `CMAKE_INSTALL_PREFIX` to use Clang's major version only. [a change]: https://reviews.llvm.org/D125860 * review: only use `llvm_version_major.sh` Since the `Makefile` can get by with only knowing Clang's major version, this change removes `llvm_version.sh` and sets `CLANG_VERSION` to use only the major part.
…ssembly#315) Sometimes a C codebase might have the odd file written in Assembly. We should force the use of Clang as the assembly compiler for these cases (which it is perfectly capable of doing), and make sure we pass on the WASI compiler triple the same way we do for C and C++.
WebAssembly/wasi-libc#403 fixed an issue with `a_barrier` that should be included in the next release of wasi-sdk. This change updates wasi-libc to the latest `HEAD` of `main` to include it.
This commit passes through `EXTRA_CFLAGS` to the libcxx build, along with `EXTRA_CXXFLAGS`. This models after the pattern in `wasi-libc` for passing these flags to thread more flags through.
This change refactors the script logic that checks if a workflow in fact built the commit matching our release tag out into a separate script. This is mainly an improvement in clarity.
When testing out these scripts on a fork, I realized that it was more convenient to use a single environment variable (e.g., `GITHUB_API_URL=... ci/draft-release.sh`) to change some of the script's input parameters.
I found the artifacts created to have incorrect version numbers; clearing the Actions cache should resolve this.
This patch enables using latest version of wasmtime for testing. This should also make it possible to running tests for wasm32-wasi-threads in the future.
This is a recommendation from WebAssembly#304 to build less of LLVM in CI. Perhaps it can speed up build times.
It looks like some macOS-specific CMake options to LLVM end up forcing an x86_64 build of some blake3 intrinsics (or something like that) which causes the build to fail on AArch64 Linux, for example. I found, though, that when removing these options it was then possible to build a toolchain for AArch64 Linux.
This changes the front-page documentation to: - use out-of-line links in more places - mention the need for `libclang_rt.builtins-wasm32.a` when using standard Clang - mention how to use a `wasi-libc` sysroot - explain the experimental status of `wasm32-wasi-threads
In WebAssembly#321, some OSX-specific `Makefile` additions to `LLVM_CMAKE_FLAGS` were skipped unless `make` is run on a Darwin OS. This allowed building wasi-sdk for aarch64. But, as reported in WebAssembly#336, this also broke arm64/x86_64 universal binaries that are built during CI. The reason for this is that CI's `main.yml` overrides `LLVM_CMAKE_FLAGS` to add caching but `make` will not append to a variable set on the command line. This changes uses the `override` keyword to append to such a variable, as suggested [here]. [here]: https://www.gnu.org/software/make/manual/html_node/Override-Directive.html
I want to include WebAssembly/wasi-libc#409 which is rather easy to hit when people are experimenting wasi-threads.
FreeBSD (like MacOS) does not support the `-executable` argument for `find`. This commit checks if the OS is FreeBSD, and if so, uses the same workaround as already in place on MacOS.
Also add some basic testing for the cmake toolchain file. Fixes: dotnet#181
No need to pass `WASI_SDK_PREFIX` value from the command line. Use `CMAKE_CURRENT_LIST_DIR` to assemble the value of `WASI_SDK_PREFIX` now.
``` error: Server does not allow request for unadvertised object f992bcc08219edb283d2ab31dd3871a4a0e8220e fatal: Fetched in submodule path 'src/config', but it did not contain f992bcc08219edb283d2ab31dd3871a4a0e8220e. Direct fetching of that commit failed. ```
* Bump llvm version to 17.0.1 * docker/Dockerfile: bump LLVM_VERSION to 17
First pass at trying to fix this...
Pulls in llvm/llvm-project#186054 and adds a regression test which previously failed. Closes WebAssembly#610
Attempting to fix WebAssembly#612
I'm shamelessly copying what [AlmaLinux does](https://git.almalinux.org/rpms/libedit/src/commit/3f0893c4cd8e0cbb2f556d2fad48326c9c037a6c/SPECS/libedit.spec#L44-L48), so let's try that... Closes WebAssembly#613
Currently LLDB binaries as-is don't work, and this should fix them.
Keeping it up-to-date
…bAssembly#621) Rework of the riscv64-linux CI build to use CMake cross-compilation on a standard `ubuntu-24.04` runner, rather than a native RISE runner. ## What changed **`ci/docker/Dockerfile.riscv64-linux`** (new): - Ubuntu 24.04 base — has `crossbuild-essential-riscv64` in its package repos - Sets `CC=riscv64-linux-gnu-gcc` / `CXX=riscv64-linux-gnu-g++` so CMake detects cross-compilation and causes LLVM to build a native `llvm-tblgen` first, then cross-compile the rest of the toolchain - Sets `CARGO_TARGET_RISCV64_UNKNOWN_LINUX_GNU_LINKER` for Rust cross-builds - `XDG_CACHE_HOME=/tmp/cache` avoids write permission issues in the container **`ci/docker-build.sh`**: - Select `ci/docker/Dockerfile.<artifact>` if it exists, fall back to the default `ci/docker/Dockerfile` - Make the wasmtime volume mount conditional on `WASI_SDK_CI_SKIP_SYSROOT != 1` **`.github/workflows/main.yml`**: - New `riscv64-linux` matrix entry: `os: ubuntu-24.04`, `rust_target: riscv64-unknown-linux-gnu` - `cross_cmake_args: -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=riscv64 -DWASI_SDK_LLDB=OFF` - `WASI_SDK_CI_SKIP_SYSROOT: 1` - Handle `cross_cmake_args` in the cmake flags step ## Why WASI_SDK_CI_SKIP_SYSROOT The cross-compiled clang runs on riscv64, not on the x86_64 build host, so the wasm sysroot step is skipped. ## Why WASI_SDK_LLDB=OFF Avoids cross-compiling libedit and libxml2 in this first iteration; can be re-enabled as a follow-up. Closes WebAssembly#607 --------- Signed-off-by: Bruno Verachten <gounthar@gmail.com>
) This commit collects together some LLVM PRs, some changes in the build configuration here, and some thoughts from WebAssembly#565 and related issues. Specifically the changes here are: * The patch for llvm/llvm-project#168449 is updated to its upstream (unlanded) form. * Patches for the (landed) llvm/llvm-project#185770 and llvm/llvm-project#185775 are added. * The `WASI_SDK_EXCEPTIONS` configuration is now either `ON`, `OFF`, or `DUAL`. The default depends on the version of Clang in use, where 23.0.0+ (which isn't released officially yet) will be `DUAL` and otherwise it's `OFF`. CI for our custom-built patched toolchain defaults to `DUAL`. * In `DUAL` mode libcxx is built twice into two different directories, once with exceptions and once without. This is supported by LLVM patches and means that Clang will select the right set of libraries based on compiler flags. The end result here is that the produced toolchain from this repository, by default, supports C++ exceptions. Additionally if exceptions-related flags are not passed then the final binary will not use C++ exceptions nor require the wasm exception-handling proposal. There's still follow-up work from WebAssembly#565, such as: * Subjectively it feels wordy to pass `-fwasm-exceptions` vs `-fexceptions`. * Personally I think `-mllvm -wasm-use-legacy-eh=false` should become the default upstream. * Subjectively I don't think that `-lunwind` should be necessary and it should be injected automatically with `-fwasm-exceptions` (or `-fexceptions`). * Shared libraries for exceptions remain disabled due to build errors I do not personally know how to resolve. I'll file follow-up issues for these once this has landed since they're more minor compared to the main body of "anything works". Closes WebAssembly#334 Closes WebAssembly#565 --------- Co-authored-by: Joel Dice <joel.dice@akamai.com>
I tried to built it locally but failed, hopefully CI can do it.
# Conflicts: # .gitmodules # docker/Dockerfile # src/llvm-project
lewing
force-pushed
the
enable-wasm-exceptions
branch
from
3596e0d to
1370489
Compare
lewing
changed the title
Pulls in some final wasip3 bits here and there.
…ebAssembly#626) This commit is a refactor of how the test suite in this repository is specified. Previously an all-encompassing `testcase.sh` script would run all tests and assert various properties using various scripts and files that were present throughout the test suite. This has a number of downsides: * With CMake configuration it's not a fan of using `glob` to discover tests. * Using scripts is generally not that portable as commands can differ between platforms or be missing entirely (e.g. Windows). * The current "filter" scripts are pretty brittle and often need changes with newer Wasmtime versions. For example some tests don't pass with more recent Wasmtime versions due to output changes. Instead tests are now specified manually in `CMakeLists.txt` rather than globbed up, and various options are plumbed through to CTest in CMake. This makes the tests literally just a single invocation of `wasmtime` where CTest has various assertions about the result of the process execution after-the-fact. This gets all tests passing with the latest version of Wasmtime and additionally updates CI as well.
Pull in recent updates
Release wasi-sdk-33
Member
Author
|
Note AI-generated update notice from GitHub Copilot CLI. Rebased forward to include the
|
Upstream wasi-libc deprecated the wasm32-wasi triple and renamed test expectation directories: expected/wasm32-wasi -> expected/wasm32-wasip1 expected/wasm32-wasi-threads -> expected/wasm32-wasip1-threads Additionally, the wasm32-wasip1-threads predefined-macros file no longer contains __wasm_bulk_memory__, __wasm_multivalue__, or __wasm_reference_types__ in the upstream baseline, so the patch needed to re-add them in their alphabetical position after __wasm_atomics__ instead of in the deprecated context lines. Resolves the wasi-sdk-ci 'git apply --ignore-whitespace eng/wasi-libc.patch' failure that blocked all 8 build legs after the wasi-sdk-33 tag bump. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Note AI-assisted fix from GitHub Copilot CLI. Pushed dc29bf6: rebase All 8 wasi-sdk-ci build legs were failing at the same step: Root cause: When this branch advanced to the
The patch added LLVM-version-specific predefined macros ( Verified |
Upstream wasi-libc PR #685 (commit 688a685, included in the wasi-libc
snapshot at the wasi-sdk-32 tag and later) replaced its Makefile with a
CMake-based build. The previous wasi-sdk.proj invoked
make -C src/wasi-libc CC=... AR=... NM=... SYSROOT=... OBJDIR=...
which fails with 'No targets specified and no makefile found' against
the new tree. All 8 wasi-sdk-ci legs were blocked by this after the
wasi-sdk-33 tag bump.
Replace each make invocation with a per-target CMake configure/build/
install cycle modeled after upstream wasi-sdk's
cmake/wasi-sdk-sysroot.cmake define_wasi_libc_sub():
cmake -G Ninja -S src/wasi-libc -B $(OBJDIR)/wasm32-wasip1
-DTARGET_TRIPLE=wasm32-wasip1 ...
cmake --build $(OBJDIR)/wasm32-wasip1 --target sysroot
cmake --install $(OBJDIR)/wasm32-wasip1
The same pattern handles the threads variant, with the THREAD_MODEL=posix
selector replaced by TARGET_TRIPLE=wasm32-wasip1-threads.
BUILD_SHARED=OFF disables .so builds so we don't need a compiler-rt
builtins library available at libc-build time (BuildCompilerRt continues
to run after BuildLibc). CHECK_SYMBOLS=OFF and BUILD_TESTS=OFF keep the
build to just the static libc the dotnet sysroot needs.
Verified locally: cmake configure + build + install on Ubuntu with
system clang-18 produces the expected libc.a, crt1.o, libpthread.a,
libsetjmp.a etc. under lib/wasm32-wasip1/.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Note AI-assisted fix from GitHub Copilot CLI. Pushed e7e8f69: migrate Why a second fix: After the wasi-libc.patch rebase (dc29bf6), the next failure was Root cause: Upstream wasi-libc PR #685 (commit Fix: Replace the two Key CMake args:
Local verification: |
wasi-libc's CMakeLists.txt requires CMake 3.26 (introduced upstream in
PR #685 when it migrated from Make to CMake). The cbl-mariner-2.0-cross-amd64
container used by wasi-sdk-ci only ships CMake 3.21.4, so the wasi-libc
cmake invocation fails immediately with:
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
CMake 3.26 or higher is required. You are running version 3.21.4
Adding an EnsureCMake target that probes the system cmake version and
downloads the Kitware portable binary release (3.31.5) into
artifacts/obj/cmake-3.31.5/ when it's too old. Local dev environments
with cmake >= 3.26 already installed skip the download. Windows builds
also skip the probe since the MSYS2 cmake there is current enough.
All BuildLibc cmake invocations now go through the resolved $(_CMakeExe)
property so the downloaded binary is used consistently for configure,
build, and install of both wasm32-wasip1 and wasm32-wasip1-threads.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
Note AI-assisted fix from GitHub Copilot CLI. Pushed 5d3c900: download portable CMake 3.31 when the system CMake is older than 3.26. Next failure after the wasi-libc CMake migration (e7e8f69): Root cause: The cbl-mariner-2.0-cross-amd64 container used by wasi-sdk-ci ships CMake 3.21.4. wasi-libc's new CMakeLists.txt requires 3.26 (introduced when upstream migrated from Make to CMake in WebAssembly/wasi-libc#685). Fix: Added an
Future cleanup: bump the container image to one with cmake 3.26+, then drop this download target. |
lewing
added a commit
to lewing/runtime
that referenced
this pull request
May 22, 2026
…ions The original PR guarded corerun's self_test() and dotenv::self_test() with #ifndef TARGET_WASI because earlier WASI SDK versions (pre-33) shipped a sysroot whose libc++abi/libunwind did not implement C++ exception ABI symbols. The PR's accompanying WASI SDK work (dotnet/wasi-sdk#239, merging through wasi-sdk-33) ships an exception-enabled sysroot, and the PR already compiles and links the runtime with -fwasm-exceptions. So try/catch/throw works in the CoreCLR PAL and host on WASI today, and the exclusion is stale. Remove the #ifndef TARGET_WASI guards around self_test() in both corerun.cpp and dotenv.cpp so the -st flag actually runs the test instead of silently returning EXIT_FAILURE. Verified end-to-end: corerun.wasm -st under nesm prints 'Self-test passed.' and exits 0, exercising the try/throw/catch path through the WASI SDK 33 libc++abi+libunwind. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two new failures surfaced in wasi-sdk-ci after the CMake-3.31 bootstrap landed: 1. "CMake Error: CMake was unable to find a build program corresponding to 'Ninja'" — the cbl-mariner-2.0-cross-amd64 CI container does not ship Ninja. wasi-libc itself doesn't require Ninja; switching the generator to 'Unix Makefiles' (which ships with the container's make package) sidesteps the dependency. cmake --build still drives the right tool via the generator selection. 2. "CMake Error: CMAKE_ASM_COMPILER not set, after EnableLanguage" — wasi-libc's CMakeLists declares ASM as a project language for the handful of wasm32 asm sources, but CMake auto-detection cannot resolve an assembler for the wasm32 target from a generic Linux image. Pass CMAKE_ASM_COMPILER pointing at the same clang as CMAKE_C_COMPILER (clang invokes its integrated assembler for asm sources, which is what wasm-ld expects downstream). Verified locally: cmake configure now completes for wasi-libc with both Unix Makefiles and the explicit ASM_COMPILER, then proceeds to build and install the static libc successfully. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The previous commit 4cb490d documented the change in the comment block but left the literal -G Ninja in the two Exec command lines (my search-and-replace was inadvertently a no-op). Fix the actual cmake invocations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Windows CI was failing with exit 127 (command not found) on cmake inside the MSYS2 bash -lc wrapper. The MSYS2 environment in the CI container has make on PATH but not cmake, so we need to bring our own. - Flip the EnsureCMake conditional to always download on Windows. - Add a PowerShell Expand-Archive step for the .zip distribution (tar is not reliable for zip extraction on Windows). - Transform the resolved cmake path into MSYS2 bash form (/c/path/to/cmake.exe) so the bash -lc 'cmake ...' invocation can locate the executable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…gure
The cbl-mariner-2.0-cross-amd64 container does not ship a host ld.
CMake's CheckCCompilerWorks test invokes a try-compile that, by default,
tries to link an executable to verify the compiler is usable. With no
host ld available the test fails:
clang: error: unable to execute command: Executable 'ld' doesn't exist!
wasi-libc is fundamentally a cross-compile to wasm32-wasip1 and doesn't
need a host linker — the produced artifacts are static archives that
will be linked later by wasm-ld during corerun.wasm assembly. Setting
CMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY tells CMake to skip the
host-link step in compiler checks, matching the standard cross-compile
pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
wasi-libc's CMake-based install puts headers and libraries under
<sysroot>/include/wasm32-wasip1/ and <sysroot>/lib/wasm32-wasip1/,
matching the modern triple. The legacy wasm32-wasi triple is now
deprecated in upstream clang and is not what wasi-libc emits.
This left the compiler-rt and libcxx builds looking in the wrong
sysroot subdirectory, so libcxx could not resolve size_t/strlen/etc.
from wasi-libc's <string.h>:
iosfwd:218:14: error: reference to unresolved using declaration
cstring:79:1: note: using declaration annotated with
'using_if_exists' here
Update three internal-only references to the new triple:
- microsoft-wasi-sdk.cmake: 'set(triple wasm32-wasip1)' — used by
compiler-rt and libcxx via CMAKE_TOOLCHAIN_FILE; sets the clang
--target so the sysroot search path resolves correctly.
- wasi-sdk.proj: LIBCXX_LIBDIR_SUFFIX and LIBCXXABI_LIBDIR_SUFFIX
updated so libcxx installs to lib/wasm32-wasip1/ matching the
rest of the sysroot layout.
The consumer-facing toolchain files (wasi-sdk.cmake,
wasi-sdk-pthread.cmake) are intentionally left at wasm32-wasi for
backwards compatibility — they are copied to the package output for
end users, not used during our build.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
cmake.exe on Windows is a native Win32 program that validates -D
argument values as filesystem paths. Even when invoked through MSYS2
bash (where the executable lookup uses the /D/a/_work form), the path
values inside -DCMAKE_C_COMPILER=..., -B ..., -DCMAKE_INSTALL_PREFIX=...
etc. must use Windows form or cmake responds with:
CMake Error at CMakeLists.txt:29 (project):
The CMAKE_C_COMPILER:
D:/a/_work/1/s/artifacts/obj/Windows_NT.x64.Debug/llvm/x64/bin/clang
is not a full path to an existing compiler tool.
Add _CMakeCompilerLLVM, _CMakeSysrootDir, _CMakeLibcBuildDir, and
_CMakeCompilerExeSuffix properties that resolve to forward-slash
Windows paths on win and plain native paths on unix, then thread them
through the wasi-libc cmake invocations. The MSYS-form _UnixBuildLLVM
etc. properties are still used by the existing compiler-rt and libcxx
targets where they're already proven to work.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
This PR was created with assistance from GitHub Copilot.
Summary
Merge upstream WebAssembly/wasi-sdk through wasi-sdk-33, bringing in C++ exception handling support and other improvements.
Key upstream changes included
-fwasm-exceptionsand exnref supportWhy
The CoreCLR WASM interpreter on WASI uses C++ exceptions (
throw/catch) for managed exception handling dispatch. Without an exception-enabled sysroot (libc++, libc++abi, libunwind), the runtime cannot be linked.Conflict resolution
.gitmodules: Added upstream'ssrc/configsubmodule, kept dotnet'ssrc/llvm-projectpointing todotnet/llvm-projectdocker/Dockerfile: Kept upstream's updated versionsrc/llvm-project: Kept dotnet's submodule referenceTesting
The resulting sysroot has been validated with the CoreCLR WASM interpreter on both nesm and wasmtime 43, with managed exception handling (try/catch/throw) working end-to-end on both legacy and exnref EH models.