Add maui desktop testing by LoopedBard3 · Pull Request #5161 · dotnet/performance

LoopedBard3 · 2026-03-18T22:26:16Z

No description provided.

New scenario that clones dotnet/maui, builds dependencies, patches in PerfLabExporter via BenchmarkDotNet.Extensions, and runs the Core, XAML, and Graphics BDN benchmark suites on desktop (Windows). Key implementation details: - Sparse-clone of dotnet/maui with depth 1 (only needed directories) - Patches Directory.Build.props to disable non-desktop TFMs so all builds (including BDN internal builds) work without mobile workloads - Removes MAUI BDN PackageReference, injects BDN.Extensions ProjectRef - Patches Program.cs with ManualConfig + PerfLabExporter - Branch mapping: net11.0 to net11.0, net10.0 to net10.0 - All heavy work in test.py to keep correlation payload small - Pipeline entries disabled until validated in CI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Move lifecycle logic (clone, patch, build, run, collect) from standalone test.py into shared/bdndesktop.py BDNDesktopHelper class, following the same pattern as AndroidInstrumentationHelper. Add BDNDESKTOP test type to const.py and runner.py subparser routing. test.py is now 16 lines, delegating to Runner(traits).run(). Tested locally: Graphics.Benchmarks ran successfully via runner path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Move clone, branch mapping, sparse checkout, and dependency build into test.py (MAUI-specific orchestrator). BDNDesktopHelper now accepts repo_dir, benchmark_projects, and disable_props as constructor params, making it reusable for any repo's BDN desktop benchmarks. Remove bdndesktop routing from runner.py and const.py since test.py drives the helper directly with its own arg parsing. Tested locally: Graphics.Benchmarks ran successfully via refactored path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Adds a new MAUI Desktop BenchmarkDotNet (BDN) scenario to the repo, including a reusable shared helper to patch/build/run external desktop benchmark suites and (optionally) collect/upload PerfLabExporter results.

Changes:

Introduces BDNDesktopHelper (shared/bdndesktop.py) to patch TFMs, inject PerfLabExporter, build, run, and collect BDN results from external repos.
Adds a new mauiDesktopBenchmarks scenario (pre/test/post scripts) plus a Helix project file to run MAUI desktop BDN suites.
Adds (currently disabled) pipeline job stubs for the new run kind in sdk-perf-jobs.yml.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
src/scenarios/shared/runner.py	Whitespace-only change in argument parsing.
src/scenarios/shared/bdndesktop.py	New shared helper for patching external repos and running/collecting BDN desktop benchmarks.
src/scenarios/mauiDesktopBenchmarks/test.py	Clones dotnet/maui, patches props, builds dependencies, runs BDN suites via shared helper.
src/scenarios/mauiDesktopBenchmarks/pre.py	Minimal pre-command script (logging only).
src/scenarios/mauiDesktopBenchmarks/post.py	Cleans up MAUI repo checkout and combined report artifact.
eng/pipelines/sdk-perf-jobs.yml	Adds new MAUI desktop BDN job entries, currently gated by `if false`.
eng/performance/maui_desktop_benchmarks.proj	New Helix project definition for MAUI desktop BDN work item.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

src/scenarios/shared/bdndesktop.py

+                    basename = os.path.basename(report_file)
+                    dest = os.path.join(upload_root, basename)
+                    shutil.copy2(report_file, dest)
+                    log.info(f'  Copied {basename} to upload root')


src/scenarios/mauiDesktopBenchmarks/test.py

src/scenarios/mauiDesktopBenchmarks/post.py

eng/pipelines/sdk-perf-jobs.yml

src/scenarios/shared/runner.py

src/scenarios/shared/bdndesktop.py

+        # Add ProjectReference to BDN.Extensions
+        item_group = ET.SubElement(root, f'{ns}ItemGroup')
+        item_group.set('Label', 'PerfLabInjected')
+        proj_ref = ET.SubElement(item_group, f'{ns}ProjectReference')
+        proj_ref.set('Include', bdn_ext_rel)
+
+        tree.write(csproj_path, xml_declaration=True, encoding='utf-8')
+        log.info(f'  Added ProjectReference: {bdn_ext_rel}')


src/scenarios/shared/bdndesktop.py

…ty, semicolon handling - Use performance.common.remove_directory() instead of shutil.rmtree for Windows long-path handling in test.py and post.py (C4, C5) - Narrow BDN PackageReference removal to exact package names instead of prefix match to preserve optional subpackages (C1) - Always emit trailing semicolon in patched Program.cs to avoid invalid C# when source pattern lacks semicolon (C2) - Add whitespace tolerance to Directory.Build.props regex and log when a requested property has zero matches (C9) - Harmonize report glob pattern to *perf-lab-report.json across collection and upload paths (C11) - Remove whitespace-only change in runner.py (C8) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove the if false guards from both the public (win-x64) and private (win-x64-viper) MAUI Desktop BenchmarkDotNet pipeline entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Helix work items don't have git on PATH. Add _find_git() to check common Windows install locations (Program Files\Git\cmd\git.exe). If git is truly unavailable, fall back to downloading the GitHub archive as a zip and extracting only the sparse checkout directories plus root-level files (Directory.Build.props, NuGet.config, etc.). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Python's bundled SSL certificates don't include the CA certs trusted by Helix machines, causing CERTIFICATE_VERIFY_FAILED. Use curl.exe (built into Windows 10+/Server 2016+) which uses the Windows certificate store instead. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Git sparse-checkout automatically includes files in parent directories of checked-out paths. The zip fallback wasn't extracting files like src/MultiTargeting.targets and src/PublicAPI.targets because they live at the src/ level, outside the specific sparse checkout subdirectories. Compute parent directories from the sparse checkout list and include files directly in those parent dirs (non-recursively). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

XAML benchmarks on the net11.0 branch have a hardcoded net10.0 TFM that causes NU1201 restore errors. Instead of failing the entire run, track which suites built successfully and only run those. Exit with error only if ALL suites fail to build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove ##[warning]/##[error] print statements since builds run inside Helix, not directly in AzDO pipeline steps. Add --iterationCount 15 and --warmupCount 3 to BDN runs. The default adaptive algorithm was running up to 100 WorkloadActual iterations per benchmark, causing the entire run to exceed the 4-hour Helix timeout before completing even the Core suite. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Align CLI args with the repo's RecommendedConfig.cs defaults: --warmupCount 1 (1 warmup is enough) --minIterationCount 15 --maxIterationCount 20 (cap adaptive algorithm) --iterationTime 250 (250ms vs BDN default 500ms) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Patch Program.cs to parse --exclusion-filter using the existing CommandLineOptions/ExclusionFilter from BDN.Extensions, stripping it from args before passing to BenchmarkSwitcher.Run(). Exclude *MauiLoggerWithLoggerMinLevelErrorBenchmarker* by default in the MAUI test.py — these emit millions of BindableProperty error log lines per iteration, bloating output and contributing to timeouts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This will be reverted after testing is validated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…hmarks" This reverts commit 42e0736.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

LoopedBard3 · 2026-03-27T18:57:52Z

/azp run

azure-pipelines · 2026-03-27T18:57:59Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

LoopedBard3 · 2026-03-27T18:58:46Z

/azp run

azure-pipelines · 2026-03-27T18:58:59Z

Azure Pipelines successfully started running 1 pipeline(s).

LoopedBard3 and others added 3 commits March 17, 2026 14:33

LoopedBard3 requested a review from Copilot March 18, 2026 22:26

LoopedBard3 self-assigned this Mar 18, 2026

Copilot AI reviewed Mar 18, 2026

View reviewed changes

matouskozak requested a review from simonrozsival March 23, 2026 19:17

LoopedBard3 requested review from simonrozsival and removed request for simonrozsival March 23, 2026 20:32

LoopedBard3 and others added 15 commits March 23, 2026 14:38

Enable MAUI Desktop BDN pipeline jobs (public and private)

9e32285

Remove the if false guards from both the public (win-x64) and private (win-x64-viper) MAUI Desktop BenchmarkDotNet pipeline entries. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Log warning listing suites that failed to build

0c08999

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Emit ##[warning]/##[error] for AzDO pipeline visibility

90b8ce9

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

TEMPORARY: disable all SDK tests except MAUI Desktop BDN benchmarks

42e0736

This will be reverted after testing is validated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Revert "TEMPORARY: disable all SDK tests except MAUI Desktop BDN benc…

02a3988

…hmarks" This reverts commit 42e0736.

Properly setup tests.

6ed899c

Make ExclusionFilter public so external projects can use it

caa1e48

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add maui desktop testing#5161

Add maui desktop testing#5161
LoopedBard3 wants to merge 18 commits intodotnet:mainfrom
LoopedBard3:AddMauiDesktopTesting

LoopedBard3 commented Mar 18, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LoopedBard3 commented Mar 27, 2026

Uh oh!

azure-pipelines bot commented Mar 27, 2026

Uh oh!

LoopedBard3 commented Mar 27, 2026

Uh oh!

azure-pipelines bot commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

LoopedBard3 commented Mar 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LoopedBard3 commented Mar 27, 2026

Uh oh!

azure-pipelines bot commented Mar 27, 2026

Uh oh!

LoopedBard3 commented Mar 27, 2026

Uh oh!

azure-pipelines bot commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants