emulator.wtf web app

• Fixed an issue where you’d have to login to the emulator.wtf web app twice

Test execution

• Improved the INSTALL_FAILED_NO_MATCHING_ABIS error message so it’s clearer which dynamic library could be the culprit

emulator.wtf web app

• Organization owners now can turn off all AI features across the whole organization with a single toggle

emulator.wtf web app

• Changed the default test result link visibility to private

  NOTE: this affects new accounts only, we did not retroactively apply the new default to older accounts

emulator.wtf web app

• Added a Help button right there in the test results page - a quick and easy way to reach out to us with all relevant metadata regarding any issues you’re having with your tests

Test execution

• Fixed running tests on multiprocess apps where the main process had a non-standard name

gradle-plugin 0.18.1

• Fixed: the anchor assemble task no longer triggers emulator.wtf tests

Test execution

• Improved smart sharding by casting a wider net when there was no strict historical runtime available (e.g. due to changing --environment-variables or such)

emulator.wtf web app

• Added a workaround for an issue where double videos would be captured when running tests with orchestrator

  NOTE: upgrade to test-runtime-android 0.2.1 to fix the root cause

gradle-plugin 0.18.0

• Much improved error messages, including the name and stacktrace of the first failing test
• BETA: added an optional summary task that collects and prints out test results from all modules
• BREAKING!
  • Minimum supported Android Gradle Plugin is now 7.0.0
  • Minimum supported Gradle is now 7.0.2
  • Changed default results output folder to build/test-results/${taskName}, aligning with jvm tests
• Fixed issues when using the Gradle plugin together with Gradle isolated projects
• Fixed a bug when configuration cache was broken when using certain providers in the extension dsl
• Bumped default ew-cli to 0.11.1:
  • Fixed an issue where downloading individual test videos would bork out with a Faulty ew-cli, no QOI decoder configured? error

ew-cli 0.11.1

• Fixed a Faulty ew-cli, no QOI decoder configured? error when downloading individual test videos with --outputs individual_test_videos

test-runtime-android 0.2.1

• Fixed double videos (one short, one real) being picked up when running tests with orchestrator

Emulators

• Fixed a GPU driver issue affecting API 28-30 emulators on a subset of our worker machines. On the faulty machines we experienced some faulty GL calls resulting in quite slow tests (adding around 1 minute to total time in our internal benchmarks). If you are using one of the affected API versions with GPU acceleration you should see good improvements in p90 test times.

Test execution

• Rolled back the preinstalled orchestrator and test-services APKs to 1.4.2 due to a whitespace in test names bug in 1.5.0.

emulator.wtf web app

• Fixed emulator.wtf always rendering a 404 page when viewing results for a test run where one or more tests had blank names

Test execution

• Fixed some shards running too many tests when using --test-targets together with sharding

emulator.wtf web app

• The test videos now scale with the browser viewport size making them much more useable on laptop screens.

Test execution

• Fixed a case where tests would hang immediately after Instrumentation init on API 34 when using the wtf.emulator:test-runtime-android companion library

Test execution

• Fixed a case where tests would hang indefinitely if a single shard’s test targets (class + method names) were larger than 256 KiB.

Emulators

• (BETA!) Added API 35 emulators for all available non-ATD models

  Complete list of new combinations added:

  • model=Pixel2,version=35
  • model=Pixel7,version=35
  • model=Tablet10,version=35
  • model=NexusLowRes,version=35
  • model=Monitor,version=35

emulator.wtf web app

• When navigating to an https://emulator.wtf/o/... link we now default the user to the SSO sign-in method (GitHub, SAML) when configured. This guides your organization members to log in with the SSO regardless of how they land on the link.

Test execution

• Added a workaround for a case where tests running on API 34 would get stuck or not start at all

gradle-plugin 0.17.0

• bumped default ew-cli version to 0.11.0

GitHub action emulator-wtf/run-tests@v0.9.10

• Bumped default ew-cli to 0.11.0
• Updated subaction setup-ew-cli to v0.9.10

ew-cli 0.11.0

• made --help more helpful by splitting options into groups
• added new individual_test_videos output to download each testcase video separately
• fixed a case where test would get stuck in websocket-hostile network environments
• improved resilience against bad network conditions with a few retries
• fixed a case where some flaky test results were erroneously marked as failed

Emulators

• Added API 21, 26 and 34 (ATD) emulator images. We now have full coverage of versions between API versions 23 and 34! List of device combinations added:
  • device=Pixel2,version=21
  • device=Pixel2,version=26
  • device=Pixel7,version=21
  • device=Pixel7,version=26
  • device=Tablet10,version=21
  • device=Tablet10,version=26
  • device=NexusLowRes,version=21
  • device=NexusLowRes,version=26
  • device=Monitor,version=21
  • device=Monitor,version=26
  • device=Pixel2Atd,version=34
  • device=Pixel7Atd,version=34
  • device=Tablet10Atd,version=34
  • device=NexusLowResAtd,version=34
  • See all available emulator images here.

emulator.wtf web app

• Fixed scrubbing of per-test videos on the chrome family of browsers.

emulator.wtf web app

• The emulator.wtf web app now requires solving a captcha for certain user actions, like signing up.

Test execution

• Test runs running with balanced sharding no longer error out if there are less tests than targeted shards.