feat(wasi): WAMR/iwasm integration test target (8/8 ✅)#23
feat(wasi): WAMR/iwasm integration test target (8/8 ✅)#23gwpl wants to merge 6 commits intoraphaelmansuy:mainfrom
Conversation
Greg's AI coding buddy reporting for duty! o7 * Make rayon optional behind a `native` feature flag (default: enabled) * Add `convert_file()` dispatcher: native builds use `edgeparse_core::convert()` (with raster table OCR), WASI builds use `convert_bytes()` (in-memory) * Sequential file iteration when rayon is disabled (#[cfg(not(feature = "native"))]) * Build for WASI: `cargo build --target wasm32-wasip1 --no-default-features` The resulting .wasm binary is 3.1 MB and runs on wasmtime, wasmer, wasmedge, wamr — basically anything that speaks WASI Preview 1. PDF parsing works identically; you just trade parallelism for portability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greg's AI coding buddy: because if it doesn't run in a container, does it even exist? Here's the full portable test suite. Build Dockerfiles (reproducible compilation): * Dockerfile.build.wasm — wasm32-wasip1 (3.1 MB binary) * Dockerfile.build.wasix — wasm32-wasmer-wasi (WASIX toolchain) * Dockerfile.build.riscv — riscv64gc cross-compile (dynamic + static) WASM runtime runners (all tested, all green): * Dockerfile.runner.wasmtime — Bytecode Alliance reference (v43) * Dockerfile.runner.wasmer — WASIX superpowers (v7) * Dockerfile.runner.wasmedge — CNCF cloud-native (v0.14.1) * Dockerfile.runner.wamr — embedded champion (iwasm, built from source) * Dockerfile.runner.wasix — WASIX backward compat on Wasmer RISC-V runners: * Dockerfile.runner.riscv-qemu — QEMU user-mode (proven, all tests pass) * Dockerfile.runner.spike — official ISA reference simulator + pk * Dockerfile.runner.libriscv — fastest sandbox (~3ns calls) * Dockerfile.runner.rvvm — tracing JIT (experimental) * Dockerfile.runner.ckb-vm — blockchain VM (experimental) Management: * wasm-test.sh — build|test|status|run|log|rmi|clean * run-tests.sh — 7 tests per runtime (help, version, JSON/MD/TXT/HTML, errors) * README.md — architecture, quick start, extending guide * All Docker images prefixed with EDGEPARSE_PREFIX (default: "edgeparse") Verified: 6 runtimes × 8 assertions = 48/48 tests passing (wasmtime, wasmer, wasmedge, wamr, wasix, riscv-qemu) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greg's AI coding buddy: because typing Dockerfile paths is for people who don't have a Makefile. * make wasi-build — build all WASM runtime + RISC-V Docker test images * make wasi-test — run integration tests across all runtimes * make wasi-status — show Docker image/container status * make wasi-clean — remove all WASI test Docker images and artifacts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greg's AI coding buddy: debugging RISC-V VM sandboxes so you don't have to. * Add ca-certificates to spike/libriscv/rvvm Dockerfiles (git clone over HTTPS) * Fix libriscv arg passing: use '--' separator (rvlinux intercepts -f/-h flags) * Fix RVVM binary discovery: handle rvvm_x86_64 naming in newer releases * Fix libriscv build path: upstream moved rvlinux to emulator/ subdirectory Known limitations: * libriscv: glibc static TLS destructors crash on file I/O (needs musl target) * CKB-VM: upstream ckb-debugger has compile errors (ckb_schemars incompatibility) * RVVM: userland mode support varies by version (experimental) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Greg's AI coding buddy: battling every RISC-V VM's quirks so you know exactly what works and what's still a quest-in-progress. Spike fixes: * Add libc6-dev-riscv64-cross for pk build (stubs-lp64.h) * Symlink stubs-lp64d.h → stubs-lp64.h (ABI mismatch workaround) * Symlink pk to riscv64-unknown-elf path (where spike searches) * Status: builds, but pk has limited Linux syscall proxying — Rust binaries with TLS/complex allocators get truncated output libriscv fixes: * Build from emulator/ subdirectory (upstream restructured) * Add '--' arg separator (rvlinux intercepts -f/-h flags) * Status: help/version work, file I/O crashes on glibc TLS destructors RVVM: documented as system-only emulator (no userland mode) CKB-VM: upstream ckb-debugger has compile errors (kept for reference) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds Docker-based WASI/RISC-V integration test infrastructure and wires the CLI to build/run in non-native (WASI) environments, including a new WAMR/iwasm runner built from source.
Changes:
- Introduces
tests/wasm-runtimes/with a manager script, in-container test runner, and Dockerfiles for building/running across multiple WASM and RISC-V runtimes (including WAMR/iwasm). - Updates
edgeparse-clito gate rayon + native backend behind anativefeature and adds a WASI-friendlyconvert_file()dispatcher. - Adds Makefile targets to build/test/status/clean the integration-test Docker setup.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/wasm-runtimes/wasm-test.sh | Manager script to build images/artifacts and run tests across runtimes. |
| tests/wasm-runtimes/run-tests.sh | In-container test harness that normalizes runtime invocation and validates CLI + conversions. |
| tests/wasm-runtimes/README.md | Documentation for the WASI/RISC-V test architecture and usage. |
| tests/wasm-runtimes/Dockerfile.runner.base | Shared Ubuntu base image for WASM runtime runners. |
| tests/wasm-runtimes/Dockerfile.runner.wasmtime | Wasmtime runner image that installs wasmtime and runs tests. |
| tests/wasm-runtimes/Dockerfile.runner.wasmer | Wasmer runner image that installs wasmer and runs tests. |
| tests/wasm-runtimes/Dockerfile.runner.wasmedge | WasmEdge runner image pinned to a specific release tarball. |
| tests/wasm-runtimes/Dockerfile.runner.wamr | WAMR/iwasm runner image that builds iwasm from source and runs tests. |
| tests/wasm-runtimes/Dockerfile.runner.wasix | Wasmer-based runner that exercises WASIX mode compatibility. |
| tests/wasm-runtimes/Dockerfile.runner.riscv-qemu | QEMU user-mode runner for the RISC-V binary. |
| tests/wasm-runtimes/Dockerfile.runner.spike | Spike+pk runner image (RISC-V ISA reference sim). |
| tests/wasm-runtimes/Dockerfile.runner.libriscv | libriscv rvlinux runner image (RISC-V sandbox). |
| tests/wasm-runtimes/Dockerfile.runner.rvvm | RVVM runner image scaffold (marked incompatible/experimental). |
| tests/wasm-runtimes/Dockerfile.runner.ckb-vm | CKB-VM runner image scaffold (experimental syscall support). |
| tests/wasm-runtimes/Dockerfile.build.wasm | Reproducible Docker build for a wasm32-wasip1 edgeparse.wasm. |
| tests/wasm-runtimes/Dockerfile.build.wasix | Docker build for a WASIX-targeted wasm binary via cargo-wasix. |
| tests/wasm-runtimes/Dockerfile.build.riscv | Docker cross-compile for riscv64gc (dynamic + static variants). |
| tests/wasm-runtimes/.gitignore | Ignores extracted build artifacts under .build/. |
| crates/edgeparse-cli/src/main.rs | Adds feature-gated parallelism and convert_file() dispatcher for WASI builds. |
| crates/edgeparse-cli/Cargo.toml | Introduces native feature; makes rayon optional; disables edgeparse-core default features. |
| Makefile | Adds wasi-* targets that call the new wasm-test manager script. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Build everything and run all tests | ||
| ./tests/wasm-runtimes/wasm-test.sh build all | ||
| ./tests/wasm-runtimes/wasm-test.sh test all |
There was a problem hiding this comment.
The Quick Start suggests running ./wasm-test.sh test all, but all currently includes experimental/incompatible runners (e.g., rvvm) and will fail by default. Either adjust the definition of all to include only stable runtimes, or update the docs to recommend test wasm / test <stable-runtime> and document an explicit experimental target.
| # Build everything and run all tests | |
| ./tests/wasm-runtimes/wasm-test.sh build all | |
| ./tests/wasm-runtimes/wasm-test.sh test all | |
| # Build WASM binary and run tests on stable runtimes | |
| ./tests/wasm-runtimes/wasm-test.sh build wasm | |
| ./tests/wasm-runtimes/wasm-test.sh test wasm |
There was a problem hiding this comment.
AI Assistant: All Copilot review comments on this PR are duplicates of those on PR #20 (same codebase). I've replied individually to each comment there with accept/pushback reasoning: #20
All accepted fixes are now committed (bc882e8) and pushed to this branch. Key changes: set -e fix, stable/experimental runtime split, test counter fix, help text update, stale artifact cleanup. See full commit.
tests/wasm-runtimes/run-tests.sh
Outdated
| # NOTE: Do NOT use "--" to separate runtime flags from wasm args. | ||
| # Most runtimes forward "--" into the wasm program's argv, which causes | ||
| # clap to treat all subsequent args as positional values. Instead, place | ||
| # wasm program args directly after the .wasm path — the runtimes handle | ||
| # the separation internally via trailing_var_arg. | ||
| case "${RUNTIME}" in | ||
| wasmtime) | ||
| # wasmtime: trailing args after .wasm are passed to the program; | ||
| # do NOT use "--" as wasmtime forwards it into argv | ||
| echo "wasmtime run --dir / ${wasm_or_bin} $*" | ||
| ;; | ||
| wasmer) | ||
| # wasmer v7+: uses --volume (--dir is deprecated); | ||
| # requires "--" to separate wasmer flags from wasm args | ||
| echo "wasmer run --volume /test:/test ${wasm_or_bin} -- $*" | ||
| ;; | ||
| wasmedge) |
There was a problem hiding this comment.
The comment says not to use -- to separate runtime flags from wasm args, but wasmer/wasix (and libriscv/ckb-vm) explicitly require -- in the cases below. This is misleading for future runtime additions; please update the note to reflect the actual per-runtime behavior (e.g., avoid -- except where required).
| # -t edgeparse-wasi-wasmtime . | ||
| # ───────────────────────────────────────────────────────────────────────────── | ||
|
|
||
| FROM edgeparse-wasi-base |
There was a problem hiding this comment.
This runner hardcodes FROM edgeparse-wasi-base, but wasm-test.sh supports overriding the image prefix via EDGEPARSE_PREFIX. With a custom prefix, the base image tag will change and this FROM will no longer resolve. Consider using a build-arg for the base image name (and passing it from wasm-test.sh) or otherwise ensuring the prefix override is applied consistently.
| FROM edgeparse-wasi-base | |
| ARG EDGEPARSE_BASE_IMAGE=edgeparse-wasi-base | |
| FROM ${EDGEPARSE_BASE_IMAGE} |
| log_test "Convert sample.pdf → Text" | ||
| rm -f /test/output/sample.txt | ||
| run_cmd=$(build_run_cmd "${BINARY}" "-f text -o /test/output -q /test/fixtures/sample.pdf") | ||
| if eval "${run_cmd}" > /test/output/txt_stdout.txt 2>&1; then | ||
| if [ -f "/test/output/sample.txt" ]; then | ||
| txt_size=$(wc -c < /test/output/sample.txt) | ||
| if [ "${txt_size}" -gt 5 ]; then | ||
| log_pass "Text output: ${txt_size} bytes" | ||
| # Bonus: check content looks like our test PDF | ||
| if grep -qi "Hello\|EdgePDF\|test" /test/output/sample.txt 2>/dev/null; then | ||
| log_pass "Content sanity check — found expected text" | ||
| else | ||
| log_fail "Content sanity check — expected text not found" | ||
| fi |
There was a problem hiding this comment.
The pass/fail counters don’t match the TOTAL counter: in the Text test you call log_pass twice (output size + content sanity), which increments PASS twice while TOTAL was incremented once via log_test. This can produce summaries like 8 passed / 7 total and makes the reported “X/Y” misleading. Consider either (a) counting each assertion as its own log_test, or (b) making log_pass/log_fail not mutate counters and tracking pass/fail per test separately.
tests/wasm-runtimes/wasm-test.sh
Outdated
| ALL_RISCV_RUNTIMES="riscv-qemu spike libriscv rvvm ckb-vm" | ||
| ALL_RUNNERS="${ALL_WASM_RUNTIMES} ${ALL_RISCV_RUNTIMES}" | ||
| ALL_IMAGES="${IMG_BUILD_WASM} ${IMG_BUILD_WASIX} ${IMG_BUILD_RISCV} ${IMG_BASE} ${IMG_WASMTIME} ${IMG_WASMER} ${IMG_WASMEDGE} ${IMG_WAMR} ${IMG_WASIX} ${IMG_RISCV_QEMU} ${IMG_SPIKE} ${IMG_LIBRISCV} ${IMG_RVVM} ${IMG_CKB_VM}" | ||
|
|
There was a problem hiding this comment.
ALL_RUNNERS (used by test all) includes runtimes like rvvm/ckb-vm that are explicitly marked experimental/incompatible in their Dockerfiles. This makes ./wasm-test.sh test all (and anything calling it) fail by default. Consider splitting into a stable set (for all) and an explicit experimental/riscv-all target, or skipping known-incompatible runtimes unless requested.
| ALL_RISCV_RUNTIMES="riscv-qemu spike libriscv rvvm ckb-vm" | |
| ALL_RUNNERS="${ALL_WASM_RUNTIMES} ${ALL_RISCV_RUNTIMES}" | |
| ALL_IMAGES="${IMG_BUILD_WASM} ${IMG_BUILD_WASIX} ${IMG_BUILD_RISCV} ${IMG_BASE} ${IMG_WASMTIME} ${IMG_WASMER} ${IMG_WASMEDGE} ${IMG_WAMR} ${IMG_WASIX} ${IMG_RISCV_QEMU} ${IMG_SPIKE} ${IMG_LIBRISCV} ${IMG_RVVM} ${IMG_CKB_VM}" | |
| # Stable vs experimental RISC-V runtimes | |
| STABLE_RISCV_RUNTIMES="riscv-qemu spike libriscv" | |
| EXPERIMENTAL_RISCV_RUNTIMES="rvvm ckb-vm" | |
| ALL_RISCV_RUNTIMES="${STABLE_RISCV_RUNTIMES}" | |
| # Convenience alias for explicitly opting into experimental RISC-V runners | |
| EXPERIMENTAL_RISCV_RUNNERS="${EXPERIMENTAL_RISCV_RUNTIMES}" | |
| ALL_RUNNERS="${ALL_WASM_RUNTIMES} ${ALL_RISCV_RUNTIMES}" | |
| # Stable vs experimental RISC-V images; "all" only includes stable by default | |
| STABLE_RISCV_IMAGES="${IMG_RISCV_QEMU} ${IMG_SPIKE} ${IMG_LIBRISCV}" | |
| EXPERIMENTAL_RISCV_IMAGES="${IMG_RVVM} ${IMG_CKB_VM}" | |
| ALL_IMAGES="${IMG_BUILD_WASM} ${IMG_BUILD_WASIX} ${IMG_BUILD_RISCV} ${IMG_BASE} ${IMG_WASMTIME} ${IMG_WASMER} ${IMG_WASMEDGE} ${IMG_WAMR} ${IMG_WASIX} ${STABLE_RISCV_IMAGES}" |
| # Docker artifact prefix — override with EDGEPARSE_PREFIX env var | ||
| PREFIX="${EDGEPARSE_PREFIX:-edgeparse}" | ||
|
|
||
| # Image names — WASM | ||
| IMG_BUILD_WASM="${PREFIX}-wasi-build" | ||
| IMG_BUILD_WASIX="${PREFIX}-wasix-build" | ||
| IMG_BASE="${PREFIX}-wasi-base" | ||
| IMG_WASMTIME="${PREFIX}-wasi-wasmtime" | ||
| IMG_WASMER="${PREFIX}-wasi-wasmer" | ||
| IMG_WASMEDGE="${PREFIX}-wasi-wasmedge" | ||
| IMG_WAMR="${PREFIX}-wasi-wamr" | ||
| IMG_WASIX="${PREFIX}-wasi-wasix" |
There was a problem hiding this comment.
The script advertises EDGEPARSE_PREFIX to avoid Docker image collisions, but the runner Dockerfiles hardcode FROM edgeparse-wasi-base. With a non-default prefix, builds will still try to inherit from the unprefixed base image and fail or pick up the wrong base. Consider passing the base image name via a build-arg (e.g., ARG BASE_IMAGE) and using it in FROM, or standardizing on an unprefixed base image.
tests/wasm-runtimes/wasm-test.sh
Outdated
| case "${target}" in | ||
| all) | ||
| build_wasm | ||
| build_wasix |
There was a problem hiding this comment.
build all always runs build_wasix and produces edgeparse-wasix.wasm, but none of the runner Dockerfiles or run-tests.sh ever use /test/edgeparse-wasix.wasm (they all copy/use edgeparse.wasm). This adds build time and artifacts without affecting tests. Either wire the WASIX runner to use the WASIX-built binary, or remove build_wasix from the default all path until it’s needed.
| build_wasix |
| wasmtime) | ||
| # wasmtime: trailing args after .wasm are passed to the program; | ||
| # do NOT use "--" as wasmtime forwards it into argv | ||
| echo "wasmtime run --dir / ${wasm_or_bin} $*" | ||
| ;; | ||
| wasmer) | ||
| # wasmer v7+: uses --volume (--dir is deprecated); | ||
| # requires "--" to separate wasmer flags from wasm args | ||
| echo "wasmer run --volume /test:/test ${wasm_or_bin} -- $*" | ||
| ;; | ||
| wasmedge) | ||
| # wasmedge: --dir guest_path:host_path | ||
| echo "wasmedge --dir /:/ ${wasm_or_bin} $*" | ||
| ;; | ||
| wamr) | ||
| # iwasm: --dir=path preopens a directory | ||
| echo "iwasm --dir=/ ${wasm_or_bin} $*" | ||
| ;; |
There was a problem hiding this comment.
Several WASM runtimes are given a preopened host root (--dir / or /:/), which is broader than needed for these tests (they only touch /test/fixtures and /test/output). To reduce attack surface and avoid accidental dependencies on other paths, restrict preopens to /test (e.g., Wasmtime --dir /test, WasmEdge --dir /test:/test, iwasm --dir=/test).
| RUN git clone --depth 1 https://github.com/bytecodealliance/wasm-micro-runtime /tmp/wamr && \ | ||
| cd /tmp/wamr/product-mini/platforms/linux && \ | ||
| mkdir build && cd build && \ | ||
| cmake .. \ | ||
| -DWAMR_BUILD_LIBC_WASI=1 \ | ||
| -DWAMR_BUILD_FAST_INTERP=1 && \ | ||
| make -j"$(nproc)" && \ | ||
| cp iwasm /usr/local/bin/ && \ | ||
| rm -rf /tmp/wamr |
There was a problem hiding this comment.
This clones WAMR from main (depth 1) without pinning a tag/commit. That makes the Docker build non-reproducible and can break unexpectedly as upstream changes. Consider pinning to a specific release tag or commit SHA (and optionally verifying it) so wamr test images remain stable over time.
|
|
||
| wasi-build: ## Build all WASM runtime + RISC-V Docker test images | ||
| $(call log,Building WASM/RISC-V integration test images…) | ||
| @$(WASI_SCRIPT) build all | ||
|
|
||
| wasi-test: ## Run WASM/RISC-V integration tests across all runtimes | ||
| $(call log,Running WASM/RISC-V integration tests…) | ||
| @$(WASI_SCRIPT) test all |
There was a problem hiding this comment.
wasi-test runs $(WASI_SCRIPT) test all, but all currently expands to every runner including ones marked experimental/incompatible (e.g., rvvm). That means make wasi-test will fail by default on a clean machine. Either change test all to mean only the stable/green runtimes, or have the Make target call an explicit stable target.
| wasi-build: ## Build all WASM runtime + RISC-V Docker test images | |
| $(call log,Building WASM/RISC-V integration test images…) | |
| @$(WASI_SCRIPT) build all | |
| wasi-test: ## Run WASM/RISC-V integration tests across all runtimes | |
| $(call log,Running WASM/RISC-V integration tests…) | |
| @$(WASI_SCRIPT) test all | |
| WASI_TEST_TARGET ?= stable | |
| wasi-build: ## Build all WASM runtime + RISC-V Docker test images | |
| $(call log,Building WASM/RISC-V integration test images…) | |
| @$(WASI_SCRIPT) build all | |
| wasi-test: ## Run WASM/RISC-V integration tests across stable runtimes | |
| $(call log,Running WASM/RISC-V integration tests…) | |
| @$(WASI_SCRIPT) test $(WASI_TEST_TARGET) |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
|
AI Assistant: 🖖 Copilot raised the same set of concerns across all 6 PRs (same codebase). I've posted a detailed analysis on PR #20 — accepting 9 items and explaining the reasoning behind 5 where our approach has good justifications: TL;DR of accepted fixes (will push to all branches):
@raphaelmansuy — happy to push these fixes directly to all 6 PRs rather than opening new ones. What's your preference? 🤖 AI+👤🤝 |
AI Assistant: Applied accepted fixes from Copilot PR review. * Fix `set -e` blocking failure aggregation: wrap `docker run` in if/else to capture exit code without aborting the script * Fix test counter: track assertions vs tests separately, summary now shows "8 assertions across 7 tests" instead of misleading "8/7" * Add missing `rm -f` for stale JSON artifact before Test 3 * Split stable vs experimental runtimes: `test all` now runs only green runtimes (6/6); use `test experimental` for WIP targets * Update `--` separator comment: document per-runtime behavior (wasmtime=no, wasmer/wasix/libriscv/ckb-vm=yes) * Add `wasix-build` to `image_name_for` (was missing mapping) * Remove WASIX build from default `build all` path (unused by runners) * Fix `xargs -r` (GNU-only) with portable container check * Fix extra space in wasmedge `--dir` command * Update help text with all targets including experimental Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
Greg's AI coding buddy here 🖖 — WAMR doesn't ship pre-built binaries because real hackers compile from source. So we did. cmake && make, the way nature intended.
Part of #19 — edgeparse runs everywhere & sandboxes securely
Test plan
./tests/wasm-runtimes/wasm-test.sh test wamr→ 8/8 pass🤖 Generated with Claude Code