feat(services): Add wasi-fs backend using wasi:filesystem interface

[JMLX42]

Summary

Adds a new wasi-fs backend that implements filesystem access via the wasi:filesystem interface for WebAssembly components running in WASI Preview 2 runtimes (wasmtime, wasmer).

Closes #7004

AI Assistance Disclosure

This implementation was developed with assistance from Claude (Anthropic). The AI helped with:

• Researching OpenDAL service patterns and WASI Preview 2 APIs
• Designing the implementation architecture
• Writing the initial implementation code
• Code review and iteration

Design Decisions

1. Crate Architecture

Decision: Split crate under core/services/wasi-fs/ (not inline in core)

Rationale: Follows OpenDAL's modern service pattern (RFC 6828). Services are being migrated to separate crates for minimal compile-time impact and independent versioning.

2. WASI Bindings Approach

Decision: Use the wasi crate (v0.14.7) with pre-generated bindings

Rationale:

• Pre-generated bindings avoid build complexity of running wit-bindgen during compilation
• Well-maintained by Bytecode Alliance, stays in sync with WASI spec
• Simpler dependency management than checking in WIT files

3. Async Model

Decision: Wrap synchronous WASI calls directly in async methods

Rationale:

• WASI Preview 2 filesystem operations are fundamentally synchronous from the component's perspective
• The WASI runtime handles actual I/O scheduling, so blocking in the component doesn't block the host thread
• No spawn_blocking needed since there's no separate thread pool in WASM

4. Preopened Directory Handling

Decision: Match root path to preopened directories from host runtime

Rationale:

• Most flexible for users - they specify the desired root path
• Builder iterates preopened directories and finds the best match
• Falls back to first preopened directory if root is / or not specified
• Provides clear error messages if no matching preopened directory exists

5. Platform Restriction

Decision: Entire crate guarded by #![cfg(all(target_arch = "wasm32", target_os = "wasi"))]

Rationale:

• Crate only makes sense on wasm32-wasip2 target
• Compiles to nothing on native targets (intentional)
• Feature flag can be safely enabled without conditional compilation issues

Implementation

Files Created

core/services/wasi-fs/
├── Cargo.toml          # Crate manifest with wasi 0.14.7 dependency
└── src/
    ├── lib.rs          # Module exports + auto-registration via #[ctor::ctor]
    ├── config.rs       # WasiFsConfig (root path option)
    ├── backend.rs      # WasiFsBuilder + WasiFsBackend (Access impl)
    ├── core.rs         # Core WASI filesystem operations
    ├── reader.rs       # oio::Read implementation
    ├── writer.rs       # oio::Write implementation
    ├── lister.rs       # oio::List implementation
    ├── deleter.rs      # oio::OneShotDelete implementation
    ├── error.rs        # WASI ErrorCode → OpenDAL ErrorKind mapping
    └── docs.md         # Service documentation

Capabilities

Capability	Supported
stat	✅
read	✅
write	✅
create_dir	✅
delete	✅
list	✅
copy	✅ (via read+write)
rename	✅
append	✅
atomic_write	❌

Limitations

• No atomic writes: abort() returns error since WASI doesn't provide atomic file operations
• No file locking: Not supported by WASI filesystem interface
• Platform-specific: Only compiles for wasm32-wasip2 target

Test Plan

• cargo check passes on native target
• cargo check --features services-wasi-fs passes
• cargo check --target wasm32-wasip2 -p opendal-service-wasi-fs passes
• cargo fmt --all passes
• Behavior tests via wasmtime (see .github/workflows/test_wasi_fs.yml)

Testing Infrastructure Added

• .env.example - Added wasi-fs configuration template
• .github/services/wasi_fs/wasi_fs/action.yml - CI service setup action
• .github/workflows/test_wasi_fs.yml - Dedicated CI workflow for wasi-fs
• core/scripts/test_wasi_fs.sh - Local test runner script

Running Tests Locally

# Prerequisites
rustup target add wasm32-wasip2
curl https://wasmtime.dev/install.sh -sSf | bash

# Build and verify service compiles
cargo check --target wasm32-wasip2 -p opendal-service-wasi-fs

Known Testing Limitation

The standard OpenDAL behavior test framework (cargo test behavior) uses tokio which doesn't compile for wasm32-wasip2. The CI workflow and test runner script are prepared for when WASI-compatible test infrastructure becomes available.

Fixes Included

• Fixed web_time cfg conditions in opendal-core/src/raw/time.rs to properly distinguish browser WASM (target_os = "unknown") from WASI (target_os = "wasi")

Usage Example

use opendal::services::WasiFs;
use opendal::Operator;

let builder = WasiFs::default().root("/data");
let op = Operator::new(builder)?.finish();

let content = op.read("hello.txt").await?;
op.write("output.txt", "Hello, WASI!").await?;

# Runtime invocation
wasmtime run --dir /host/path::/guest/path component.wasm

[JMLX42]

WASI Usability Investigation

While implementing integration tests, I discovered a fundamental architectural issue with using the wasi-fs service.

The Problem

The wasi-fs service implementation is correct - WasiFsCore works perfectly with blocking WASI Preview 2 APIs. However, there's no way to use it through OpenDAL's public API because the entire Operator infrastructure depends on tokio:

1. Async Operator - Requires tokio runtime to poll futures
2. blocking::Operator - Requires tokio::runtime::Handle to call block_on:

// From core/core/src/blocking/operator.rs
pub struct Operator {
    handle: tokio::runtime::Handle,  // Requires tokio!
    op: AsyncOperator,
}

impl Operator {
    pub fn new(op: AsyncOperator) -> Result<Self> {
        Ok(Self {
            handle: Handle::try_current()  // Needs tokio runtime
                .map_err(|_| Error::new(...))?,
            op,
        })
    }
}

Tokio doesn't compile for wasm32-wasip2:

error: Only features sync,macros,io-util,rt,time are supported on wasm.

Current State

• ✅ WasiFsCore - Works (synchronous, uses WASI Preview 2 APIs directly)
• ❌ Operator - Cannot be used (requires async runtime)
• ❌ blocking::Operator - Cannot be used (requires tokio handle)

Possible Solutions

1. Expose WasiFsCore as public API - Simple but breaks consistency with other services
2. WASI-specific Operator - New operator type that doesn't require tokio
3. Alternative async runtime for WASI - Use smol or async-executor instead of tokio
4. Native blocking trait methods - Let wasi-fs implement blocking methods directly without wrapping

Questions for Maintainers

• Is WASI support a priority for OpenDAL?
• What's the preferred approach to make wasi-fs actually usable?
• Should we hold this PR until there's a path to usability?

The service implementation is complete and correct, but without a usable API, it's effectively dead code.

[JMLX42]

Update: WASIp3 is Now Available

After further investigation, WASI 0.3 (WASIp3) with native async support is now available:

• Wasmtime v39.0.0 (November 20, 2025) includes initial WASIp3 support
• Native async is implemented via the "cooperative multithreading component model proposal"
• The Component Model now supports future<T> and stream<T> types natively

What This Means

Instead of building workarounds for WASIp2's lack of async, the wasi-fs service could be updated to target WASIp3:

Approach	Target	Async Support	Operator Compatibility
Current	WASIp2 (wasm32-wasip2)	None - blocking only	❌ Requires tokio
Updated	WASIp3	Native async in Component Model	✅ Could work with standard Operator

Recommendation

Consider updating wasi-fs to target WASIp3 instead of building a separate WasiBlockingOperator. This would:

1. Use the standard Operator API (no special WASI path)
2. Leverage native async support in the Component Model
3. Align with the future direction of WASI

The tradeoff is that WASIp3 is newer, so runtime support may be less widespread than WASIp2. However, Wasmtime (the most common WASI runtime) already supports it.

References:

• Wasmtime v39.0.0 Release Notes
• Looking Ahead to WASIp3 - Fermyon

[Xuanwo]

Thank you for working on this!

[Xuanwo]

It's better to not use scripts. We can just move those logic to github workflows.

[JMLX42]

It's better to not use scripts. We can just move those logic to github workflows.

@Xuanwo you mean inlining the script into the github workflow?

[Xuanwo]

We don't need to manually simulate copying if it lacks native copy support.

[JMLX42]

OK I'll remove the custom copy impl.

[JMLX42]

OK I'll remove the custom copy impl.

Just to add a bit more context:

1. WASI doesn't have native copy (no copy_file_range equivalent)
2. OpenDAL doesn't provide a copy fallback - if a backend doesn't support copy, users get an error
3. The fs service uses tokio::fs::copy which delegates to the OS (which may use copy_file_range on Linux or a read/write loop internally - but that's the OS's responsibility, not OpenDAL's)

IMHO, if point 3 stands for fs, it does sort of stand for wasi too. But the lack of customizable buffer size is a problem all by itself. Which is a good enough a reason to keep this a user level job and remove copy().

[JMLX42]

Fixed in 01f2bec

[Xuanwo]

Maybe we can just return self here?

[JMLX42]

Thanks for the review! I investigated the WASI Preview 2 specification.

From the wasi-filesystem WIT spec, the read-directory function explicitly states:

"On filesystems where directories contain entries referring to themselves and their parents, often named . and .. respectively, these entries are omitted."

This is different from Preview 1 which included them. So:

1. The . and .. check is actually dead code - they'll never be returned
2. The returned_self approach is necessary since we won't get . from WASI
3. The suggestion to "return self here" won't work because . never appears

I'll remove the unnecessary ./.. check but keep the returned_self logic.

---

Disclosure: This analysis was performed with assistance from Claude Code (AI).

[JMLX42]

Fixed in 681baa1 - removed the unnecessary check and simplified the match expression.

[Xuanwo]

We only have blocking API here?

[JMLX42]

Yes, WASI Preview 2 only provides blocking APIs - this is a platform limitation.

Comparison

	Standard OpenDAL	WASIp2 (current)	WASIp3 (future)
Async Runtime	tokio	None (blocking only)	Component Model native (future<T>, stream<T>)
Operator	Operator	Needs WasiOperator (blocking)	WasiOperator (async)
Compiles for wasm32-wasip2	❌	✅	N/A
Compiles for wasm32-wasip3	❌	✅	✅

Current State (WASIp2)

To make wasi-fs usable at the Operator level today, OpenDAL would need a WasiOperator that doesn't depend on tokio. This operator would use blocking calls internally since that's all WASIp2 offers.

Future Enhancement (WASIp3)

WASI 0.3 (available in Wasmtime v39+) introduces native async support via future<T> and stream<T> in the Component Model. A future WasiOperator could leverage WASIp3's async primitives instead of blocking calls, aligning with OpenDAL's async-first architecture without requiring tokio.

---

Disclosure: This response was drafted with assistance from Claude Code (AI).