Investigate and speed up Android instrumentation tests

[andrewtavis]

Terms

• I have searched open and closed feature requests
• I agree to follow Scribe-Android's Code of Conduct

Description

As of now the instrumentation tests that run via .github/workflows/android_ci.yml#L90 are taking quite some time to finish. This issue would look into potential ways to speed them up.

Contribution

Happy to discuss possible solutions and review a PR when one is opened! 🚀

[Femimat]

Hi @andrewtavis, may I take a look at this issue?

[andrewtavis]

Sounds good, @Femimat, but could we finalize #573 first? We can assign you here after that 😊

[andrewtavis]

Assigning as #573 is stalled a bit. Feel free to work on both of these at once, @Femimat!

[Femimat]

Thanks, @andrewtavis I’ll look into both issues.

[Femimat]

Hi, @andrewtavis here are some questions I wanted to ask about the issue

Parallelisation & Resource Limits: I’m planning to use a matrix strategy to run Conjugate and Keyboards tests in parallel. Does the organisation have any specific limits on the number of concurrent GitHub Actions runners we should be aware of?

Delays & Stability: I noticed there are several sleep commands (totaling 40s) in the current script. I’ve removed them in favor of relying on the emulator runner's built-in wait logic. Were these sleeps added to fix specific flakiness on GitHub's runners that I should be careful about?

Build vs. Emulator Contention: I have separated the APK build step from the emulator script so they don't compete for CPU resources. This usually speeds things up, but it does change the log output sequence. Is this separation preferred?

API Level: The current tests run on API 29. Would you be open to testing on a different API level if it proves to be more stable or faster on CI?

I have optimised the instrumentation tests by implementing a matrix strategy to run the Conjugate and Keyboards flavors in parallel. I also decoupled the APK assembly from the emulator execution to reduce resource contention and removed redundant sleep delays. These changes should significantly speed up the CI pipeline while improving stability."

[andrewtavis]

Hi @Femimat 👋 Sorry for the wait in the reply here.

Parallelisation & Resource Limits: I’m planning to use a matrix strategy to run Conjugate and Keyboards tests in parallel. Does the organisation have any specific limits on the number of concurrent GitHub Actions runners we should be aware of?

I wouldn't see this as being a problem :)

Delays & Stability: I noticed there are several sleep commands (totaling 40s) in the current script. I’ve removed them in favor of relying on the emulator runner's built-in wait logic. Were these sleeps added to fix specific flakiness on GitHub's runners that I should be careful about?

You'd be welcome to remove them and then we can see if it works!

Build vs. Emulator Contention: I have separated the APK build step from the emulator script so they don't compete for CPU resources. This usually speeds things up, but it does change the log output sequence. Is this separation preferred?

I think your change sounds fine :) We can see how it works in a PR!

API Level: The current tests run on API 29. Would you be open to testing on a different API level if it proves to be more stable or faster on CI?

100% would be ok :)

Feel free to open a PR!

[andrewtavis]

Closed by #595 🚀 Thanks for the fix, @Femimat!