Skip to content

[Testing] Consider increasing timeouts for PreformanceTest #11031

@simonrozsival

Description

@simonrozsival

The Xamarin.Android.Build.Tests.PerformanceTest tests are flaky:

Build_AndroidManifest_Change:

Exceeded expected time of 4500ms, actual 4653.098ms
   at Xamarin.Android.Build.Tests.PerformanceTest.Profile(ProjectBuilder builder, Int32 iterations, Action`1 action, Action`1 afterRun, String caller) in /Users/builder/azdo/_work/4/s/android/tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs:line 74
   at Xamarin.Android.Build.Tests.PerformanceTest.Profile(ProjectBuilder builder, Action`1 action, String caller) in /Users/builder/azdo/_work/4/s/android/tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs:line 53
   at Xamarin.Android.Build.Tests.PerformanceTest.Build_AndroidManifest_Change() in /Users/builder/azdo/_work/4/s/android/tests/MSBuildDeviceIntegration/Tests/PerformanceTest.cs:line 265
   at InvokeStub_PerformanceTest.Build_AndroidManifest_Change(Object, Object, IntPtr*)

I've seen this and other similar test fail quite frequently. I understand the need for these tests to identify major perf regressions we might introduce, but they should not fail often. I have these suggestions:

  • Increase Retry count (currently 2),
  • or bump the expected times.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Area: Unit TestsIssues with repo unit tests.needs-triageIssues that need to be assigned.

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions