Improve VirtioFs performance with swiotlb by asherkariv · Pull Request #40518 · microsoft/WSL

asherkariv · 2026-05-13T01:18:24Z

Summary

Improve VirtioFs performance by adding swiotlb (Software Translation Lookaside Buffer) bounce buffer support for virtio devices. Swiotlb enables more efficient DMA handling for VirtioFs, reducing overhead in file system operations.

PR Checklist

Builds clean (Release x64)
Code review passed

Detailed Description

Commit 1: Add experimental.swiotlb .wslconfig setting

Adds a new [experimental] config key swiotlb to specify custom swiotlb configuration
Adds localization strings and validation

Commit 2: Swiotlb gating, vmId plumbing, and devicehost override

Swiotlb gating for VirtioFs perf: Automatically compute default swiotlb bounce buffer config when VirtioFs (or other virtio devices) are enabled, enabling efficient DMA for file system operations
Virtio9p deprecation ordering: Move the virtio9p deprecation/disable block before swiotlb computation so deprecated devices don't trigger unnecessary bounce buffer config
vmId plumbing: Inject vmId= into all virtio device options via \GuestDeviceManager::AddHdvShareWithOptions\ for proper device-to-VM association
WSL_DEVICE_HOST_DLL override: Add CMake option to replace wsldevicehost.dll without a NuGet update (dev-loop improvement)
Fix empty-options guard: Prevent double semicolons when appending to device options
Package update: Update packages.config

Validation Steps

Built successfully in Release x64 configuration
Code reviewed for logic correctness (swiotlb gating order verified)

Copilot

Pull request overview

This PR introduces a new experimental .wslconfig key (experimental.swiotlb) to control SWIOTLB bounce buffer behavior for WSL VMs using virtio devices. The setting is validated during config parsing, optionally propagated to the guest kernel command line (range mode), and also forwarded to wsldevicehost.dll via a virtiofs “name-with-options” control token.

Changes:

Add experimental.swiotlb parsing/validation and store both a virtiofs token value (SwiotlbCfg) and an optional kernel cmdline value (SwiotlbKernelCfg).
Plumb SWIOTLB configuration into virtiofs share options and (range mode) append hv_pci_swiotlb= to the guest kernel command line.
Add new constants and localization strings for invalid-value and virtiofs-required warnings.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/windows/service/inc/wslservice.idl	Adds `LXSS_VM_KERNEL_SWIOTLB_CONFIG` constant used when building the kernel cmdline.
src/windows/service/exe/WslCoreVm.cpp	Appends SWIOTLB kernel parameter (range mode) and adds virtiofs control token in share options.
src/windows/common/WslCoreConfig.h	Adds config key name and new config fields (`SwiotlbCfg`, `SwiotlbKernelCfg`) + telemetry presence bit.
src/windows/common/WslCoreConfig.cpp	Implements `experimental.swiotlb` parsing/validation and enforces virtiofs dependency.
src/shared/inc/lxinitshared.h	Adds `LX_INIT_SWIOTLB_MOUNT_OPTIONS` mount-option/control-token prefix.
localization/strings/en-US/Resources.resw	Adds localized warning strings for invalid SWIOTLB and virtiofs-required cases.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

benhillis · 2026-05-13T23:57:12Z

I might be confused but I thought we didn’t need this setting until we have the updated kernel?

…n WSLC A user-supplied WSL2 kernel without the WSL swiotlb patch ignores hv_pci_swiotlb= but still receives the wsldevicehost `;swiotlb=` tokens from PR #40518, mis-configuring DMA. Detect kernel support at runtime in mini_init via env-var presence (unrecognized key=value cmdline params are passed to init as env vars; early_param/__setup consumes them when the patch is present). Report the bit back through LX_INIT_GUEST_CAPABILITIES and cache the effective swiotlb value on WslCoreVm right after ReadGuestCapabilities so both consumer sites (drvfs virtiofs and VirtioProxy networking) reuse it and the kernel-unsupported warning fires at most once. Move drvfs thread spawn after ReadGuestCapabilities so the flag is available without a wait/event dance. Enable swiotlb unconditionally on WSLC since it ships its own patched kernel. Hoist c_swiotlbDefault to GuestDeviceManager.h and fold the swiotlb=force hv_pci_swiotlb=<cfg> append into AppendCommonKernelCommandLine. Replace the ad-hoc EnableVirtioFs/VirtioProxy consumer detection with the EnableVirtio master switch, which also covers WSLg's virtio-fs device. Reject NetworkingMode=VirtioProxy when EnableVirtio=false with a fallback to NAT and a user warning. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

- Add experimental.swiotlb .wslconfig setting for custom bounce buffer configuration - Gate swiotlb default computation: only compute when VirtioFs, Virtio9p, or VirtioProxy networking is enabled (both WSL2 and WSLC paths) - Move virtio9p deprecation before swiotlb computation so disabled devices don't trigger unnecessary bounce buffer config - Inject vmId=<machineId> into all virtio device options via GuestDeviceManager::AddHdvShareWithOptions for device-to-VM association - Add WSL_DEVICE_HOST_DLL CMake override to replace wsldevicehost.dll without a NuGet update (dev-loop improvement) - Fix empty-options guard to prevent double semicolons - Add KernelSupportsHvPciSwiotlb to guest capabilities - Add localization strings and validation for swiotlb config - Update packages.config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Use a 1 GiB base for the default hv_pci_swiotlb reservation and require enough guest memory for the full reserved range before enabling it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

ptrivedi · 2026-05-26T19:24:28Z

    <comment>{FixedPlaceholder="{}"}Command line arguments, file names and string inserts should not be translated</comment>
  </data>
+  <data name="MessageConfigInvalidSwiotlb" xml:space="preserve">
+    <value>Invalid SWIOTLB value '{}' for key '{}' in {}:{}. Expected format: '0x&lt;hex&gt;,&lt;size&gt;K|M'</value>


nit: Looks like your regex for the format allows for case-insensitive k|m as well. It lists only the uppercase as the expected format in other places in the code as well

Correct, the example uses the more common use of K,M, but we still do allow a lowercase postfix.

ptrivedi · 2026-05-26T19:58:34Z

+    // Reserve a swiotlb bounce buffer for virtio devices.
+    if (!swiotlbConfig.empty())
+    {
+        kernelCmdLine += std::format(L" swiotlb=force hv_pci_swiotlb={}", swiotlbConfig);


Don't you need a sizing parameter for swiotlb (especially) along with 'force'? Without the sizing parameter, I am reading that it will default to 64MB floor on x86_64 systems (a bit better on arm where it tries to cap the automatic allocation at a maximum of 6% of total system RAM or a hard ceiling). I do feel that's low and will cause you to run out of swiotlb memory...

For this feature, we are using the kernel "swiotlb engine", and providing our own swiotlb pool, which the kernel uses for hv_pci devices. The hv_pci swiotlb pool parameters are specified via hv_pci_swiotlb=,.
Thus, we can leave the swiotlb default region to use the default size.

ptrivedi · 2026-05-26T22:43:07Z

+
+    // Receive and parse the guest kernel version
+    ReadGuestCapabilities();
+


Moving this before drvfs initialization would affect our startup latency? Before, the drvfs initialization could have proceeded concurrently with the kernel boot

ptrivedi · 2026-05-26T22:48:59Z

+
+    // 1 GiB base sits below 4 GiB (32-bit PCI BAR limit on ARM64 virtio), below the Hyper-V
+    // low PCI MMIO hole (starts at >= 3 GiB on both x64 and ARM64), and above early boot allocations.
+    return L"0x40000000,64M";


why do we return a hardcoded string here instead of using the c_swiotlbBase and c_swiotlbSize constants to construct it?

ptrivedi · 2026-05-26T22:56:47Z

+        {
+            try
+            {
+                wil::com_ptr<IPlan9FileSystem> instance;


Hmm, is this a fix for an existing issue? or needed for swiotlb buffer release or something?

ptrivedi · 2026-05-26T23:00:34Z


    MESSAGE_HEADER Header;
    bool SeccompAvailable;
+    bool KernelSupportsHvPciSwiotlb;


thinking out loud here. We shouldn't have to worry about backward compat when adding to this struct, because init and service will always be deployed together...?

This is for the case where the user uses a private kernel, that may or may not have the path.

Yes wslservice and init always come from the same build so this isn't an issue

OneBlue · 2026-05-27T17:36:06Z

        ConfigKey(ConfigSetting::Experimental::HostAddressLoopback, EnableHostAddressLoopback),
-        ConfigKey(ConfigSetting::Experimental::SetVersionDebug, SetVersionDebug)};
+        ConfigKey(ConfigSetting::Experimental::SetVersionDebug, SetVersionDebug),
+        ConfigKey(ConfigSetting::Experimental::Swiotlb, std::move(parseSwiotlb))};


What's the thinking for exposing a new .wslconfig value for this ? Could we set a hardcoded value first ? That would greatly simplify this change

asherkariv · 2026-05-27T17:58:55Z

Canceling this PR, as we have an updated PR using a different kernel patch, which lets the wsldevicehost know where the swiotlb region has been allocated.
#40654

Copilot AI review requested due to automatic review settings May 13, 2026 01:18

Copilot started reviewing on behalf of asherkariv May 13, 2026 01:19 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread src/windows/common/WslCoreConfig.cpp Outdated

Comment thread src/windows/common/WslCoreConfig.cpp Outdated

benhillis reviewed May 13, 2026

View reviewed changes

Comment thread src/shared/inc/lxinitshared.h Outdated

benhillis reviewed May 13, 2026

View reviewed changes

Comment thread src/windows/common/WslCoreConfig.cpp

benhillis reviewed May 13, 2026

View reviewed changes

Comment thread src/windows/service/inc/wslservice.idl Outdated

Copilot AI review requested due to automatic review settings May 13, 2026 21:49

Copilot started reviewing on behalf of asherkariv May 13, 2026 21:52 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread src/windows/common/WslCoreConfig.h Outdated

benhillis requested a review from Copilot May 13, 2026 23:52

Copilot started reviewing on behalf of benhillis May 13, 2026 23:53 View session

Copilot AI reviewed May 13, 2026

View reviewed changes

Comment thread test/windows/UnitTests.cpp Outdated

Copilot AI review requested due to automatic review settings May 18, 2026 21:05

Copilot started reviewing on behalf of benhillis May 18, 2026 21:07 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Comment thread src/windows/service/exe/WslCoreVm.cpp Outdated

benhillis force-pushed the user/askariv/swiotlb-enable branch from 643d5f7 to 31ef8c1 Compare May 19, 2026 01:45

Copilot AI review requested due to automatic review settings May 19, 2026 22:27

benhillis force-pushed the user/askariv/swiotlb-enable branch from 31ef8c1 to 0468b35 Compare May 19, 2026 22:27

Copilot started reviewing on behalf of benhillis May 19, 2026 22:33 View session

benhillis force-pushed the user/askariv/swiotlb-enable branch from 146ca92 to 5664a8f Compare May 20, 2026 19:01

benhillis changed the title ~~wslservice: Add experimental.swiotlb .wslconfig setting~~ wslservice: Add swiotlb support with gating, vmId plumbing, and devicehost override May 20, 2026

benhillis reviewed May 20, 2026

View reviewed changes

Comment thread src/windows/common/helpers.cpp

benhillis force-pushed the user/askariv/swiotlb-enable branch 5 times, most recently from b273696 to 6468f4e Compare May 22, 2026 19:58

benhillis changed the title ~~wslservice: Add swiotlb support with gating, vmId plumbing, and devicehost override~~ wslservice: Improve VirtioFs performance with swiotlb, vmId plumbing, and devicehost override May 22, 2026

benhillis changed the title ~~wslservice: Improve VirtioFs performance with swiotlb, vmId plumbing, and devicehost override~~ Improve VirtioFs performance with swiotlb May 22, 2026

benhillis marked this pull request as ready for review May 22, 2026 20:00

benhillis requested a review from a team as a code owner May 22, 2026 20:00

benhillis force-pushed the user/askariv/swiotlb-enable branch from 6468f4e to 77a20c1 Compare May 22, 2026 20:03

Ben Hillis and others added 3 commits May 26, 2026 10:28

Move default hv_pci_swiotlb reservation

8fe484d

Use a 1 GiB base for the default hv_pci_swiotlb reservation and require enough guest memory for the full reserved range before enabling it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

adjust virtio options and device host teardown

ebd7a34

benhillis force-pushed the user/askariv/swiotlb-enable branch from 5555239 to ebd7a34 Compare May 26, 2026 17:29

ptrivedi reviewed May 26, 2026

View reviewed changes

benhillis force-pushed the user/askariv/swiotlb-enable branch from 18acc78 to ebd7a34 Compare May 27, 2026 00:34

OneBlue reviewed May 27, 2026

View reviewed changes

asherkariv closed this May 27, 2026


		// Receive and parse the guest kernel version
		ReadGuestCapabilities();

Conversation

asherkariv commented May 13, 2026 • edited by benhillis Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

PR Checklist

Detailed Description

Commit 1: Add experimental.swiotlb .wslconfig setting

Commit 2: Swiotlb gating, vmId plumbing, and devicehost override

Validation Steps

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

benhillis commented May 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asherkariv commented May 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

asherkariv commented May 13, 2026 •

edited by benhillis

Loading