feat(wasi): Wasmer integration test target (8/8 ✅) by gwpl · Pull Request #21 · raphaelmansuy/edgeparse

gwpl · 2026-03-25T10:22:11Z

Summary

Greg's AI coding buddy here 🖖 — Wasmer's --dir is deprecated, --volume is the new hotness. We figured that out so you don't have to.

Adds Wasmer v7.x as an integration test target (the only runtime with WASIX support)
Uses --volume for directory mapping (v7+ compatible)
8/8 tests passing: full PDF extraction pipeline verified

Part of #19 — edgeparse runs everywhere & sandboxes securely

Test plan

./tests/wasm-runtimes/wasm-test.sh test wasmer → 8/8 pass

🤖 Generated with Claude Code

Greg's AI coding buddy reporting for duty! o7 * Make rayon optional behind a `native` feature flag (default: enabled) * Add `convert_file()` dispatcher: native builds use `edgeparse_core::convert()` (with raster table OCR), WASI builds use `convert_bytes()` (in-memory) * Sequential file iteration when rayon is disabled (#[cfg(not(feature = "native"))]) * Build for WASI: `cargo build --target wasm32-wasip1 --no-default-features` The resulting .wasm binary is 3.1 MB and runs on wasmtime, wasmer, wasmedge, wamr — basically anything that speaks WASI Preview 1. PDF parsing works identically; you just trade parallelism for portability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Greg's AI coding buddy: because if it doesn't run in a container, does it even exist? Here's the full portable test suite. Build Dockerfiles (reproducible compilation): * Dockerfile.build.wasm — wasm32-wasip1 (3.1 MB binary) * Dockerfile.build.wasix — wasm32-wasmer-wasi (WASIX toolchain) * Dockerfile.build.riscv — riscv64gc cross-compile (dynamic + static) WASM runtime runners (all tested, all green): * Dockerfile.runner.wasmtime — Bytecode Alliance reference (v43) * Dockerfile.runner.wasmer — WASIX superpowers (v7) * Dockerfile.runner.wasmedge — CNCF cloud-native (v0.14.1) * Dockerfile.runner.wamr — embedded champion (iwasm, built from source) * Dockerfile.runner.wasix — WASIX backward compat on Wasmer RISC-V runners: * Dockerfile.runner.riscv-qemu — QEMU user-mode (proven, all tests pass) * Dockerfile.runner.spike — official ISA reference simulator + pk * Dockerfile.runner.libriscv — fastest sandbox (~3ns calls) * Dockerfile.runner.rvvm — tracing JIT (experimental) * Dockerfile.runner.ckb-vm — blockchain VM (experimental) Management: * wasm-test.sh — build|test|status|run|log|rmi|clean * run-tests.sh — 7 tests per runtime (help, version, JSON/MD/TXT/HTML, errors) * README.md — architecture, quick start, extending guide * All Docker images prefixed with EDGEPARSE_PREFIX (default: "edgeparse") Verified: 6 runtimes × 8 assertions = 48/48 tests passing (wasmtime, wasmer, wasmedge, wamr, wasix, riscv-qemu) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Greg's AI coding buddy: because typing Dockerfile paths is for people who don't have a Makefile. * make wasi-build — build all WASM runtime + RISC-V Docker test images * make wasi-test — run integration tests across all runtimes * make wasi-status — show Docker image/container status * make wasi-clean — remove all WASI test Docker images and artifacts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Greg's AI coding buddy: debugging RISC-V VM sandboxes so you don't have to. * Add ca-certificates to spike/libriscv/rvvm Dockerfiles (git clone over HTTPS) * Fix libriscv arg passing: use '--' separator (rvlinux intercepts -f/-h flags) * Fix RVVM binary discovery: handle rvvm_x86_64 naming in newer releases * Fix libriscv build path: upstream moved rvlinux to emulator/ subdirectory Known limitations: * libriscv: glibc static TLS destructors crash on file I/O (needs musl target) * CKB-VM: upstream ckb-debugger has compile errors (ckb_schemars incompatibility) * RVVM: userland mode support varies by version (experimental) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Greg's AI coding buddy: battling every RISC-V VM's quirks so you know exactly what works and what's still a quest-in-progress. Spike fixes: * Add libc6-dev-riscv64-cross for pk build (stubs-lp64.h) * Symlink stubs-lp64d.h → stubs-lp64.h (ABI mismatch workaround) * Symlink pk to riscv64-unknown-elf path (where spike searches) * Status: builds, but pk has limited Linux syscall proxying — Rust binaries with TLS/complex allocators get truncated output libriscv fixes: * Build from emulator/ subdirectory (upstream restructured) * Add '--' arg separator (rvlinux intercepts -f/-h flags) * Status: help/version work, file I/O crashes on glibc TLS destructors RVVM: documented as system-only emulator (no userland mode) CKB-VM: upstream ckb-debugger has compile errors (kept for reference) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Adds Docker-based integration testing for running edgeparse under multiple WASM runtimes (incl. Wasmer v7+) and several RISC-V sandboxes, plus a small CLI feature split so the binary can be built for WASI without native-only deps.

Changes:

Introduces tests/wasm-runtimes/ test manager + in-container test runner scripts.
Adds build + runner Dockerfiles for WASI/WASIX and RISC-V targets.
Updates edgeparse-cli to gate rayon/“native” behavior behind a native feature (default-on), enabling --no-default-features WASI builds; wires Makefile targets.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 21 comments.

Show a summary per file

File	Description
tests/wasm-runtimes/wasm-test.sh	Docker build/test manager for the runtime matrix
tests/wasm-runtimes/run-tests.sh	In-container test runner that normalizes runtime CLI differences
tests/wasm-runtimes/README.md	Documentation for the new WASM/RISC-V integration test infrastructure
tests/wasm-runtimes/Dockerfile.runner.wasmtime	Wasmtime runner image to execute the WASI binary
tests/wasm-runtimes/Dockerfile.runner.wasmer	Wasmer runner image (WASI) to execute the WASI binary
tests/wasm-runtimes/Dockerfile.runner.wasmedge	WasmEdge runner image to execute the WASI binary
tests/wasm-runtimes/Dockerfile.runner.wasix	Wasix-mode runner image (via Wasmer)
tests/wasm-runtimes/Dockerfile.runner.wamr	WAMR/iwasm runner image built from source
tests/wasm-runtimes/Dockerfile.runner.riscv-qemu	QEMU user-mode runner for the RISC-V dynamic ELF
tests/wasm-runtimes/Dockerfile.runner.spike	Spike+pk runner for RISC-V static ELF
tests/wasm-runtimes/Dockerfile.runner.libriscv	libriscv/rvlinux runner for RISC-V static ELF
tests/wasm-runtimes/Dockerfile.runner.rvvm	RVVM runner (experimental) for RISC-V static ELF
tests/wasm-runtimes/Dockerfile.runner.ckb-vm	CKB-VM runner (experimental) for RISC-V static ELF
tests/wasm-runtimes/Dockerfile.runner.base	Shared Ubuntu base layer for WASM runtime runners
tests/wasm-runtimes/Dockerfile.build.wasm	Reproducible WASI build image producing `edgeparse.wasm`
tests/wasm-runtimes/Dockerfile.build.wasix	Reproducible WASIX build image producing `edgeparse-wasix.wasm`
tests/wasm-runtimes/Dockerfile.build.riscv	Reproducible RISC-V cross-build producing dynamic + static ELFs
tests/wasm-runtimes/.gitignore	Ignores extracted build artifacts under `.build/`
crates/edgeparse-cli/src/main.rs	Adds `convert_file()` dispatcher + feature-gated parallelism for WASI builds
crates/edgeparse-cli/Cargo.toml	Introduces `native` feature (default) and makes rayon optional; disables edgeparse-core default features
Makefile	Adds `wasi-build`, `wasi-test`, `wasi-status`, `wasi-clean` targets

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T13:20:07Z

tests/wasm-runtimes/Dockerfile.build.wasix

+# cargo-wasix installs its own rustup toolchain + wasm32-wasmer-wasi target
+RUN cargo install cargo-wasix


This Dockerfile installs cargo-wasix without pinning a version. Since cargo-wasix and the underlying toolchain/target can change, builds may become non-reproducible or start failing unexpectedly. Consider pinning cargo-wasix (e.g. cargo install cargo-wasix --version ...) and/or the Rust toolchain used in this builder stage.

Suggested change

# cargo-wasix installs its own rustup toolchain + wasm32-wasmer-wasi target

RUN cargo install cargo-wasix

# Pin cargo-wasix version for reproducible builds (override via --build-arg if needed)

ARG CARGO_WASIX_VERSION=0.15.16

# cargo-wasix installs its own rustup toolchain + wasm32-wasmer-wasi target

RUN cargo install cargo-wasix --version "${CARGO_WASIX_VERSION}"

AI Assistant: All Copilot review comments on this PR are duplicates of those on PR #20 (same codebase). I've replied individually to each comment there with accept/pushback reasoning: #20

All accepted fixes are now committed (bc882e8) and pushed to this branch. Key changes: set -e fix, stable/experimental runtime split, test counter fix, help text update, stale artifact cleanup. See full commit.

Copilot · 2026-03-25T13:20:07Z

tests/wasm-runtimes/Dockerfile.runner.wasix

+
+FROM edgeparse-wasi-base
+
+RUN curl https://get.wasmer.io -sSfL | sh


This installs Wasmer via curl ... | sh without pinning a version, so the wasix runner may drift away from the intended Wasmer v7.x baseline over time. Consider pinning the Wasmer version here as well (and ideally verifying downloads) to keep the test target stable.

Suggested change

RUN curl https://get.wasmer.io -sSfL | sh

RUN WASMER_VERSION=v7.0.0 curl https://get.wasmer.io -sSfL | sh

Copilot · 2026-03-25T13:20:07Z

tests/wasm-runtimes/wasm-test.sh

+# Docker artifact prefix — override with EDGEPARSE_PREFIX env var
+PREFIX="${EDGEPARSE_PREFIX:-edgeparse}"
+
+# Image names — WASM
+IMG_BUILD_WASM="${PREFIX}-wasi-build"
+IMG_BUILD_WASIX="${PREFIX}-wasix-build"
+IMG_BASE="${PREFIX}-wasi-base"
+IMG_WASMTIME="${PREFIX}-wasi-wasmtime"
+IMG_WASMER="${PREFIX}-wasi-wasmer"
+IMG_WASMEDGE="${PREFIX}-wasi-wasmedge"
+IMG_WAMR="${PREFIX}-wasi-wamr"
+IMG_WASIX="${PREFIX}-wasi-wasix"


The script advertises EDGEPARSE_PREFIX as a safe way to avoid Docker image name collisions, but the runner Dockerfiles currently hard-code FROM edgeparse-wasi-base, so overriding the prefix will break builds. Either fully support the prefix by parameterizing the base image tag (build-arg) or remove the prefix override feature/documentation.

Copilot · 2026-03-25T13:20:07Z

tests/wasm-runtimes/Dockerfile.runner.wamr

+#                -t edgeparse-wasi-wamr .
+# ─────────────────────────────────────────────────────────────────────────────
+
+FROM edgeparse-wasi-base


This Dockerfile hard-codes FROM edgeparse-wasi-base, but wasm-test.sh supports a configurable EDGEPARSE_PREFIX that changes the base image tag. That combination will break builds when the prefix is overridden. Consider adding ARG BASE_IMAGE + FROM ${BASE_IMAGE} and passing --build-arg BASE_IMAGE=$(IMG_BASE) from the manager script.

Suggested change

FROM edgeparse-wasi-base

ARG BASE_IMAGE=edgeparse-wasi-base

FROM ${BASE_IMAGE}

Copilot · 2026-03-25T13:20:08Z

tests/wasm-runtimes/Dockerfile.runner.wasmtime

+RUN curl -sSf https://wasmtime.dev/install.sh | bash
+ENV PATH="/root/.wasmtime/bin:${PATH}"
+


This installs Wasmtime by piping a remote script into bash and implicitly pulls the latest version. That makes the integration tests non-reproducible and can break unexpectedly when upstream changes. Consider pinning the Wasmtime version (e.g. via an ARG/ENV), and prefer downloading a specific release artifact with checksum verification over curl | bash.

Suggested change

RUN curl -sSf https://wasmtime.dev/install.sh | bash

ENV PATH="/root/.wasmtime/bin:${PATH}"

ARG WASMTIME_VERSION=22.0.0

ARG WASMTIME_ARCH=x86_64

RUN set -eux; \

curl -sSfL "https://github.com/bytecodealliance/wasmtime/releases/download/v${WASMTIME_VERSION}/wasmtime-v${WASMTIME_VERSION}-${WASMTIME_ARCH}-linux.tar.xz" -o /tmp/wasmtime.tar.xz; \

curl -sSfL "https://github.com/bytecodealliance/wasmtime/releases/download/v${WASMTIME_VERSION}/wasmtime-v${WASMTIME_VERSION}-${WASMTIME_ARCH}-linux.tar.xz.sha256" -o /tmp/wasmtime.tar.xz.sha256; \

sha256sum -c /tmp/wasmtime.tar.xz.sha256; \

mkdir -p /opt/wasmtime; \

tar -xJf /tmp/wasmtime.tar.xz -C /opt/wasmtime --strip-components=1; \

rm /tmp/wasmtime.tar.xz /tmp/wasmtime.tar.xz.sha256

ENV PATH="/opt/wasmtime/bin:${PATH}"

Copilot · 2026-03-25T13:20:10Z

tests/wasm-runtimes/README.md

+## Docker Image Naming
+
+All images are prefixed with `edgeparse` (configurable via `EDGEPARSE_PREFIX`):
+
+* `edgeparse-wasi-build` — WASM build environment
+* `edgeparse-riscv-build` — RISC-V cross-compilation environment
+* `edgeparse-wasi-base` — Shared runner base
+* `edgeparse-wasi-wasmtime` — Wasmtime runner
+* `edgeparse-wasi-wasmer` — Wasmer runner


This section claims image names are prefixed with edgeparse and configurable via EDGEPARSE_PREFIX, but the current runner Dockerfiles hard-code FROM edgeparse-wasi-base, so a non-default prefix won’t actually work. Update either the Dockerfiles/build logic to support a configurable base image tag, or adjust the documentation to match reality.

Copilot · 2026-03-25T13:20:11Z

tests/wasm-runtimes/run-tests.sh

+# ─────────────────────────────────────────────────────────────────────────────
+# TEST 1: --help flag works
+# ─────────────────────────────────────────────────────────────────────────────
+log_test "CLI --help flag"
+run_cmd=$(build_run_cmd "${BINARY}" "--help")
+if eval "${run_cmd}" > /test/output/help.txt 2>&1; then
+    if grep -qi "edgeparse" /test/output/help.txt; then
+        log_pass "Help output contains 'edgeparse'"
+    else
+        log_fail "Help output doesn't mention 'edgeparse'"
+    fi
+else
+    log_fail "Exit code non-zero: ${run_cmd}"
+fi
+
+# ─────────────────────────────────────────────────────────────────────────────
+# TEST 2: --version flag works
+# ─────────────────────────────────────────────────────────────────────────────
+log_test "CLI --version flag"
+run_cmd=$(build_run_cmd "${BINARY}" "--version")
+if eval "${run_cmd}" > /test/output/version.txt 2>&1; then
+    if grep -qE '[0-9]+\.[0-9]+\.[0-9]+' /test/output/version.txt; then
+        version=$(cat /test/output/version.txt)
+        log_pass "Version: ${version}"


The PR description mentions “8/8 tests passing”, but this runner currently defines 7 top-level tests (log_test is called 7 times). If 8 is the intended number, consider adding the missing test (or counting the content sanity check as its own test) and making the total consistent across docs/output.

Copilot · 2026-03-25T13:20:11Z

tests/wasm-runtimes/Dockerfile.runner.wasmedge

+FROM edgeparse-wasi-base
+
+# Pin version for reproducibility; update when needed
+ARG WASMEDGE_VERSION=0.14.1
+


This Dockerfile hard-codes FROM edgeparse-wasi-base, but the manager script supports an EDGEPARSE_PREFIX override that changes the base image tag. Builds will break when the prefix is not edgeparse. Consider parameterizing the base image via a build arg and passing it from wasm-test.sh.

Suggested change

FROM edgeparse-wasi-base

# Pin version for reproducibility; update when needed

ARG WASMEDGE_VERSION=0.14.1

ARG EDGEPARSE_PREFIX=edgeparse

FROM ${EDGEPARSE_PREFIX}-wasi-base

# Pin version for reproducibility; update when needed

ARG WASMEDGE_VERSION=0.14.1

Copilot · 2026-03-25T13:20:11Z

tests/wasm-runtimes/wasm-test.sh

+#   ./wasm-test.sh build   [all|wasm|riscv|base|wasmtime|wasmer|wasmedge|wamr|riscv-qemu]
+#   ./wasm-test.sh test    [all|wasmtime|wasmer|wasmedge|wamr|riscv-qemu]


Copilot · 2026-03-25T13:20:11Z

tests/wasm-runtimes/run-tests.sh

+        wasmtime)
+            # wasmtime: trailing args after .wasm are passed to the program;
+            # do NOT use "--" as wasmtime forwards it into argv
+            echo "wasmtime run --dir / ${wasm_or_bin} $*"
+            ;;
+        wasmer)
+            # wasmer v7+: uses --volume (--dir is deprecated);
+            # requires "--" to separate wasmer flags from wasm args
+            echo "wasmer run --volume /test:/test ${wasm_or_bin} -- $*"
+            ;;
+        wasmedge)
+            # wasmedge: --dir guest_path:host_path
+            echo "wasmedge --dir /:/  ${wasm_or_bin} $*"
+            ;;
+        wamr)
+            # iwasm: --dir=path preopens a directory
+            echo "iwasm --dir=/ ${wasm_or_bin} $*"
+            ;;


The WASI runners preopen the container’s entire root (/) into the WASM sandbox (e.g. --dir / / --dir /:/). That’s much broader than needed for these tests (they only require /test), and it weakens the sandboxing guarantees this infrastructure is meant to validate. Consider preopening only /test (and possibly /test/fixtures + /test/output) for each runtime.

raphaelmansuy · 2026-03-25T13:28:22Z

@copilot open a new pull request to apply changes based on the comments in this thread

gwpl · 2026-03-25T14:41:35Z

AI Assistant: 🖖

Copilot raised the same set of concerns across all 6 PRs (same codebase). I've posted a detailed analysis on PR #20 — accepting 9 items and explaining the reasoning behind 5 where our approach has good justifications:

👉 Full response on PR #20

TL;DR of accepted fixes (will push to all branches):

Fix set -e blocking failure aggregation
Fix test counter mismatch (8 passed / 7 total)
Add missing rm -f for stale JSON artifact
Split stable vs experimental runtimes (test all = stable only)
Update help text and -- separator docs
Remove unused WASIX build from default path

@raphaelmansuy — happy to push these fixes directly to all 6 PRs rather than opening new ones. What's your preference?

🤖 AI+👤🤝

AI Assistant: Applied accepted fixes from Copilot PR review. * Fix `set -e` blocking failure aggregation: wrap `docker run` in if/else to capture exit code without aborting the script * Fix test counter: track assertions vs tests separately, summary now shows "8 assertions across 7 tests" instead of misleading "8/7" * Add missing `rm -f` for stale JSON artifact before Test 3 * Split stable vs experimental runtimes: `test all` now runs only green runtimes (6/6); use `test experimental` for WIP targets * Update `--` separator comment: document per-runtime behavior (wasmtime=no, wasmer/wasix/libriscv/ckb-vm=yes) * Add `wasix-build` to `image_name_for` (was missing mapping) * Remove WASIX build from default `build all` path (unused by runners) * Fix `xargs -r` (GNU-only) with portable container check * Fix extra space in wasmedge `--dir` command * Update help text with all targets including experimental Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

gwpl and others added 5 commits March 25, 2026 08:45

gwpl mentioned this pull request Mar 25, 2026

edgeparse sandboxes securely & runs everywhere — WASM/RISC-V integration test infrastructure #19

Open

14 tasks

raphaelmansuy requested a review from Copilot March 25, 2026 13:11

Copilot started reviewing on behalf of raphaelmansuy March 25, 2026 13:11 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

		# cargo-wasix installs its own rustup toolchain + wasm32-wasmer-wasi target
		RUN cargo install cargo-wasix


		FROM edgeparse-wasi-base

		RUN curl https://get.wasmer.io -sSfL \| sh

	RUN curl https://get.wasmer.io -sSfL \| sh
	RUN WASMER_VERSION=v7.0.0 curl https://get.wasmer.io -sSfL \| sh

	FROM edgeparse-wasi-base
	ARG BASE_IMAGE=edgeparse-wasi-base
	FROM ${BASE_IMAGE}

		RUN curl -sSf https://wasmtime.dev/install.sh \| bash
		ENV PATH="/root/.wasmtime/bin:${PATH}"

-RUN curl -sSf https://wasmtime.dev/install.sh | bash
-ENV PATH="/root/.wasmtime/bin:${PATH}"
+ARG WASMTIME_VERSION=22.0.0
+ARG WASMTIME_ARCH=x86_64
+RUN set -eux; \
+    curl -sSfL "https://github.com/bytecodealliance/wasmtime/releases/download/v${WASMTIME_VERSION}/wasmtime-v${WASMTIME_VERSION}-${WASMTIME_ARCH}-linux.tar.xz" -o /tmp/wasmtime.tar.xz; \
+    curl -sSfL "https://github.com/bytecodealliance/wasmtime/releases/download/v${WASMTIME_VERSION}/wasmtime-v${WASMTIME_VERSION}-${WASMTIME_ARCH}-linux.tar.xz.sha256" -o /tmp/wasmtime.tar.xz.sha256; \
+    sha256sum -c /tmp/wasmtime.tar.xz.sha256; \
+    mkdir -p /opt/wasmtime; \
+    tar -xJf /tmp/wasmtime.tar.xz -C /opt/wasmtime --strip-components=1; \
+    rm /tmp/wasmtime.tar.xz /tmp/wasmtime.tar.xz.sha256
+ENV PATH="/opt/wasmtime/bin:${PATH}"

		# ./wasm-test.sh build [all\|wasm\|riscv\|base\|wasmtime\|wasmer\|wasmedge\|wamr\|riscv-qemu]
		# ./wasm-test.sh test [all\|wasmtime\|wasmer\|wasmedge\|wamr\|riscv-qemu]

Conversation

gwpl commented Mar 25, 2026

Summary

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

gwpl Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

raphaelmansuy commented Mar 25, 2026

Uh oh!

gwpl commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants