ew-cli 0.12.0

• Added the ability to pass secret runner arguments via --secret-environment-variables, for use with things like tokens, credentials, passwords, etc. These will be passed to the test runner in the same way as --environment-variables, but won’t be visible in the emulator.wtf UI. Alternatively, secrets can be passed in via environment variables having the EW_SECRET_ prefix, e.g. EW_SECRET_foo=bar.
• You can now override the test instrumentation runner used with --test-runner-class.
• Fixed formatting issues when displaying --help.
• Added support for tunneling emulator egress networking access over to ew-cli. Read more about the egress tunnel here.

Test execution

• Fixed a rare case where tests would fail with “Invalid test apk” with certain incrementally built apks emitted by the Android Gradle Plugin.

emulator.wtf web app

• Test classes and methods can now be sorted in the test results page.
• Added indicators to class names to reflect their expanded/collapsed state.

emulator.wtf web app

• The test results page now shows test runtimes for all classes and methods

emulator.wtf web app

• The test results page now has a light mode available
• Improved how the test runs list is rendered on narrower screens

emulator.wtf web app

• Fixed rendering long test run display names in the test runs list

Test execution

• Fixed a very rare case where the emulator would get stuck for multiple minutes before running the tests

emulator.wtf web app

• Fixed an issue where you’d have to login to the emulator.wtf web app twice

Test execution

• Improved the INSTALL_FAILED_NO_MATCHING_ABIS error message so it’s clearer which dynamic library could be the culprit

emulator.wtf web app

• Organization owners now can turn off all AI features across the whole organization with a single toggle

emulator.wtf web app

• Changed the default test result link visibility to private

  NOTE: this affects new accounts only, we did not retroactively apply the new default to older accounts

emulator.wtf web app

• Added a Help button right there in the test results page - a quick and easy way to reach out to us with all relevant metadata regarding any issues you’re having with your tests

Test execution

• Fixed running tests on multiprocess apps where the main process had a non-standard name

gradle-plugin 0.18.1

• Fixed: the anchor assemble task no longer triggers emulator.wtf tests

Test execution

• Improved smart sharding by casting a wider net when there was no strict historical runtime available (e.g. due to changing --environment-variables or such)

emulator.wtf web app

• Added a workaround for an issue where double videos would be captured when running tests with orchestrator

  NOTE: upgrade to test-runtime-android 0.2.1 to fix the root cause

gradle-plugin 0.18.0

• Much improved error messages, including the name and stacktrace of the first failing test
• BETA: added an optional summary task that collects and prints out test results from all modules
• BREAKING!
  • Minimum supported Android Gradle Plugin is now 7.0.0
  • Minimum supported Gradle is now 7.0.2
  • Changed default results output folder to build/test-results/${taskName}, aligning with jvm tests
• Fixed issues when using the Gradle plugin together with Gradle isolated projects
• Fixed a bug when configuration cache was broken when using certain providers in the extension dsl
• Bumped default ew-cli to 0.11.1:
  • Fixed an issue where downloading individual test videos would bork out with a Faulty ew-cli, no QOI decoder configured? error

ew-cli 0.11.1

• Fixed a Faulty ew-cli, no QOI decoder configured? error when downloading individual test videos with --outputs individual_test_videos

test-runtime-android 0.2.1

• Fixed double videos (one short, one real) being picked up when running tests with orchestrator

Emulators

• Fixed a GPU driver issue affecting API 28-30 emulators on a subset of our worker machines. On the faulty machines we experienced some faulty GL calls resulting in quite slow tests (adding around 1 minute to total time in our internal benchmarks). If you are using one of the affected API versions with GPU acceleration you should see good improvements in p90 test times.

Test execution

• Rolled back the preinstalled orchestrator and test-services APKs to 1.4.2 due to a whitespace in test names bug in 1.5.0.

emulator.wtf web app

• Fixed emulator.wtf always rendering a 404 page when viewing results for a test run where one or more tests had blank names

Test execution

• Fixed some shards running too many tests when using --test-targets together with sharding

emulator.wtf web app

• The test videos now scale with the browser viewport size making them much more useable on laptop screens.

Test execution

• Fixed a case where tests would hang immediately after Instrumentation init on API 34 when using the wtf.emulator:test-runtime-android companion library

Test execution

• Fixed a case where tests would hang indefinitely if a single shard’s test targets (class + method names) were larger than 256 KiB.

Emulators

• (BETA!) Added API 35 emulators for all available non-ATD models

  Complete list of new combinations added:

  • model=Pixel2,version=35
  • model=Pixel7,version=35
  • model=Tablet10,version=35
  • model=NexusLowRes,version=35
  • model=Monitor,version=35

emulator.wtf web app

• When navigating to an https://emulator.wtf/o/... link we now default the user to the SSO sign-in method (GitHub, SAML) when configured. This guides your organization members to log in with the SSO regardless of how they land on the link.

Test execution

• Added a workaround for a case where tests running on API 34 would get stuck or not start at all