Part 3[CI GitHub Actions and Bitrise] Android Automation Testing with Gradle Managed Devices using Firebase TestLab devices

How to configure Firebase test lab Gradle managed devices to run on GitHub Actions and Bitrise.

In the previous posts run GMD from Android Studio and run GMD using Firebase test lab devices from Android Studio we configure how to run GMD locally, in this tutorial we are going to configure GMD to run on our CI pipelines. We are going to focus on setting up Gradle Managed Devices, to run on continuous integration pipelines using Github Actions and Bitrise using FTL devices.

Goal

Setup Github Actions and Bitrise to run GMD tasks.

1. Github Actions

Prerequisites

• Push Android project code to Github.
• Firebase Test Lab account.
• Use the latest Android Studio (at this time Hedgehog canary version)
• Use the latest Gradle managed devices is table in AGP (Android Gradle Plugin) 8.0.+
• Configure GitHub Actions: Add GitHub actions to Android project [follow this tutorial: GitHub Actions Android — Tutorial how to build Android app using GitHub Actions as CI]. After your project is configure to use Github Actions continue the steps below.

Steps

1. Add your Firebase Test lab project service account credential to your Android Studio project. Go to Google cloud to get service account credential JSON file. Add the file to your repo. Example: (../app/AccountCredentials.json). Note it’s not recommded to push secrets to a public Github repo, to secure your credentials I recommend storing them on Github secret. The tutorial https://readmedium.com/android-development-how-to-secure-service-account-credentials-json-file-bccd71ab17ed shows how to uses Github secrets so you don’t need to push your credentials to Github.
2. Edit the file “build.gradle”(app module)

Configure the Firebase Test Lab Gradle managed device.

firebaseTestLab {

        serviceAccountCredentials.set(file("service-account.json"))

        managedDevices {
            create("myFtlDevice") {
                device = "panther" // Pixel 7
                apiLevel = 33
            }
        }
    } 

3. Add below to the file gradle.properties

android.experimental.testOptions.managedDevices.customDevice=true

*Now we completed the setup for Firebase TestLab. Next step is to configure Github Actions yaml file to run the Gradle task.

./gradlew myFtlDeviceDebugAndroidTest

Above runs the configuration locally.

Below steps show how to add the Gradle task “./gradlew myFtlDeviceDebugAndroidTest” to run FTL GMD using GitHub actions

1. Go to the file android.yml. Enter below code:

run_GMD:
 runs-on: macos-latest

 steps:
   - name: List Android Packages
     run: sudo $ANDROID_HOME/tools/bin/sdkmanager --list | sed -n '/Available Packages/q;p'
   - name: Accept license 34.0.0-rc3
     run: echo "y" | sudo $ANDROID_HOME/tools/bin/sdkmanager "build-tools;34.0.0-rc3"
   - uses: actions/checkout@v3
   - name: set up JDK 17
     uses: actions/setup-java@v3
     with:
       java-version: '17'
       distribution: 'zulu'
       cache: gradle

   - name: Grant execute permission for gradlew
     run: chmod +x gradlew

   - name: Run compose test using Firebase test lab Gradle Managed devices
     run: ./gradlew myFtlDeviceDebugAndroidTest // <----- FTL Gradle task

   - name: Upload Gradle Managed device test results
     uses: actions/upload-artifact@v3.1.1
     with:
       name: testResults
       path: app/build/reports/androidTests/managedDevice/allDevices/index.html

GMD configuration to use Gradle virtual device

testOptions {
    managedDevices {
        devices {
            device1(com.google.firebase.testlab.gradle.ManagedDevice) {
                device = "Pixel2"
                apiLevel = 30
            }
            pixel6api31(com.android.build.api.dsl.ManagedVirtualDevice) {
                // Use device profiles you typically see in Android Studio.
                device = "Pixel 6"
                // Use only API levels 30 and higher.
                apiLevel = 31
                // To include Google services, use "google".
                systemImageSource = "aosp"
            }
        }
    }
}

Gradle task

./gradlew pixel6api31DebugAndroidTest

Noticed we just need to change the Gradle task

run_GMD:
 runs-on: macos-latest

 steps:
   - name: List Android Packages
     run: sudo $ANDROID_HOME/tools/bin/sdkmanager --list | sed -n '/Available Packages/q;p'
   - name: Accept license 34.0.0-rc3
     run: echo "y" | sudo $ANDROID_HOME/tools/bin/sdkmanager "build-tools;34.0.0-rc3"
   - uses: actions/checkout@v3
   - name: set up JDK 17
     uses: actions/setup-java@v3
     with:
       java-version: '17'
       distribution: 'zulu'
       cache: gradle

   - name: Grant execute permission for gradlew
     run: chmod +x gradlew
   - name: Run compose test using Gradle Managed device
     run: ./gradlew pixel6api31DebugAndroidTest -Pandroid.testoptions.manageddevices.emulator.gpu="swiftshader_indirect"
   - name: Upload Gradle Managed device test results
     uses: actions/upload-artifact@v3.1.1
     with:
       name: testResults
       path: app/build/reports/androidTests/managedDevice/allDevices/index.html

2. Create a commit and push the code.

3. Now when you open a GitHub Pull Request, you will see the GHA check run for every PR, if the checks fails you are not able to merge code into the branch 🙂

2. Bitrise

1. Go to Bitrise.io
2. Configure new Android project. New workflow.
3. From the above steps the Android project already has the Gradle task to run gradle managed device.
4. Add build step to run Gradle Command

./gradlew pixel2api30DebugAndroidTest

5. Run the Bitrise workflow

Conclusion

TL;DR: GMD with Firebase test lab configuration runs your tests on real and virtual Android devices and generates a test results report. Using the Android Gradle managed device plugin makes UI testing faster and easier.

With the new Gradle plugin now you will have your test devices configuration and tests all in the same Android project code base. With this is easier to test on real device and you don’t have to create and maintain Android emulators, they are created and maintained by Firebase Test Lab.

Thanks for spending your time reading it and let me know if I’m wrong somewhere or if there’s something that could do differently or better. I’m open to your feedback 🙌🏻

-Andres

Relates Articles

Part 1: Run Gradle Managed Devices locally from Android Studio

Part 2: Run GMD using FTL devices locally from Android Studio

Part 3: Run Gradle Managed Devices using Firebase TestLab devices on Github or Bitrise

Part 4: Run test on FTL using GCloud command (wip)

Reference:

GitHub Actions Android — Tutorial how to build Android app using GitHub Actions as CI

Android CI with Bitrise— Tutorial how to build Android app using Bitrise as CI

Github Action notes:

Part A. Setup Github Actions on your Android project repo

1. Create a Github repo with Android app code.
2. Go to your GitHub repo and select the top tab “Actions”. Search “Android CI”.

a. Click the button “Configure”.

b. GitHub automatically generates the yml file “android.yml”. The config file builds your Android application APK. It runs the Gradle task $./gradle build. (note: the default machine used is an ubuntu machine, at the moment we can’t run GMD on linux machines because of a virtualization issue, we need to update the android.yml file to use a mac-os machine.

c. Click the button “Start commit”.

d. Commit new file.

e. It pushes the file “android.yml” to GitHub.

3. Pull the code locally, now you have the file android.yml on your repo.

4. Now the repo has a root yaml file that configures the GitHub Actions. Now when you open a PR the Github Action to build your APK gets triggered.