Docs
Blog Status Toggle Dark/Light/Auto modeToggle Dark/Light/Auto modeToggle Dark/Light/Auto modeBack to homepage

GitHub Action

There’s an official GitHub Action that can be used to run instrumentation tests as part of your GitHub Actions workflow.

Quick example

A really basic example building an app apk, test apk, running tests and then finally collecting the results with the mikepenz/action-junit-report@v2 action:

name: Run tests
on: push
jobs:
  run-tests:
    runs-on: ubuntu-20.04
  steps:
    - uses: actions/checkout@v2
    - name: Build app
      run: ./gradlew assembleDebug assembleAndroidTest
    - name: Run tests
      uses: emulator-wtf/run-tests@master
      with:
        api-token: ${{ secrets.EW_API_TOKEN }}
        app: app/build/outputs/apk/debug/app-debug.apk
        test: app/build/outputs/apk/androidTest/app-debug-androidTest.apk
        outputs-dir: build/test-results
    - name: Publish test report
      uses: mikepenz/action-junit-report@v2
      if: always() # always run even if the tests fail
      with:
        report_paths: 'build/test-results/**/*.xml'
As a best practice, do not store the emulator.wtf API token directly in your GitHub Action Workflow configuration.
Use Encrypted secrets instead.

Inputs

VariableDescription
versionew-cli version to use (Optional. Defaults to the latest version)
api-tokenAPI token for emulator.wtf. We recommend using Encrypted secrets for this.
appPath to application apk file
testPath to test apk file
outputs-dirLocation to store test outputs in
devicesDevice configurations to use, in the form of model=X,version=Y per line
use-orchestratorWhether to use the Android Test Orchestrator
clear-package-dataWhether to clear app data between every test (requires use-orchestrator)
with-coverageSet to true to collect coverage files and save them to outputs-dir
additional-apksAdditional apks to install, one per line
environment-variablesEnvironment variables to pass to AndroidJUnitRunner, one per line in the form of key=value
num-uniform-shardsSet to a number larger than 1 to randomly split your tests into multiple shards to be executed in parallel
num-shardsSet to a number larger than 1 to split your tests into multiple shards based on test counts to be executed in parallel (this results in faster total test execution time than using the num-uniform-shards flag above)
directories-to-pullDirectories to pull from device and store in outputs-dir, one per line

Common examples

Run tests with multiple device profiles

By default emulator.wtf runs tests on a Pixel2-like emulator with API 27 (Android 8.1). If you want to run on a different version or device profile you can use the devices input to do so:

    - name: Run tests
      uses: emulator-wtf/run-tests@master
      with:
        api-token: ${{ secrets.EW_API_TOKEN }}
        app: app/build/outputs/apk/debug/app-debug.apk
        test: app/build/outputs/apk/androidTest/app-debug-androidTest.apk
        devices: |
          model=NexusLowRes,version=23
          model=Pixel2,version=27          

Run tests with orchestrator while clearing package data

You can use Android Test Orchestrator to run the tests - this will create a new app VM from scratch for each test. Slower to run, but will ensure no static state leakage between tests. Add the optional clear-package-data flag to clear app persisted state between each run. Read more about orchestrator here.

    - name: Run tests
      uses: emulator-wtf/run-tests@master
      with:
        api-token: ${{ secrets.EW_API_TOKEN }}
        app: app/build/outputs/apk/debug/app-debug.apk
        test: app/build/outputs/apk/androidTest/app-debug-androidTest.apk
        use-orchestrator: true
        clear-package-data: true

Grab coverage data

Use the with-coverage flag to capture test run coverage data and store the results (one or more .exec or .ec files) in the path specified by outputs-dir:

    - name: Run tests
      uses: emulator-wtf/run-tests@master
      with:
        api-token: ${{ secrets.EW_API_TOKEN }}
        app: app/build/outputs/apk/debug/app-debug.apk
        test: app/build/outputs/apk/androidTest/app-debug-androidTest.apk
        with-coverage: true
        outputs-dir: build/test-results

Run tests with shards

The following example runs tests in parallel using 10 separate shards and stores the outputs from each shard in a separate folder under build/test-results:

    - name: Run tests
      uses: emulator-wtf/run-tests@master
      with:
        api-token: ${{ secrets.EW_API_TOKEN }}
        app: app/build/outputs/apk/debug/app-debug.apk
        test: app/build/outputs/apk/androidTest/app-debug-androidTest.apk
        num-shards: 10
        outputs-dir: build/test-results