Compose Screenshot Testing – from Setup to CI/CD Integration

flowchart TD;
    A["Create directory structure for screenshot tests"]
    B["Implement Screenshot Test"]
    C["Run Gradle task to generate baseline images"]
    D["Run tests to validate screenshots"]
    
    A --> B
    B --> C
    C --> D

1. Setup and Requirements

Apply plugin and import dependencies from Android documentation, if you need additional reference you can check the changes from this Pull Request highlights the changes accurately.

I’m currently using below version.

[versions]
screenshot = "0.0.1-alpha07"

These are the minimum requirements according to documentation.

• Kotlin 1.9.20 or higher.
• Android Gradle 8.5.0-beta01 or higher.

2. Implementing Compose Screenshot Tests

2.1 Environment Setup & Directory Structure

Create a source set for screenshot tests in your module’s directory structure as below.
This isolated source keeps screenshot test code separate from unit and instrumented tests (If you have any).

app/
 └── src/
      ├── main/
      └── screenshotTest/
           └── kotlin/
                └── com/
                     └── developersbreach/
                          └── composeactors/
                               └── screenshotTests/
                                    └── SearchScreenScreenshotTest.kt

2.2 Creating your Screenshot test from Preview

We can reuse the existing preview defined in our composable screens which are created during development.
By directly calling preview function you ensure that the exact UI rendered during development is captured for comparison for future changes as reference.

package com.developersbreach.composeactors.screenshotTests

import androidx.compose.runtime.Composable
import androidx.compose.ui.tooling.preview.Preview
import com.developersbreach.composeactors.ui.screens.search.SearchScreenUILightPreview

@Preview
@Composable
private fun SearchScreenUIPreviewTest() {
    SearchScreenUILightPreview()
}

Preview which is part of your composable screen in SearchScreenUI :

// SearchScreenUI.kt :
@Composable
fun SearchScreenUI(
    navigateUp: () -> Unit,
    navigateToSearchBySearchType: (Int) -> Unit,
    searchHint: String,
    onSearchQueryChange: (query: String) -> Unit,
    data: SearchDataType,
) {
    // Screen contents
}

@Preview(showBackground = true)
@Composable
fun SearchScreenUILightPreview() {
    ComposeActorsTheme(darkTheme = false) {
        SearchScreenUI(
            navigateUp = {},
            navigateToSearchBySearchType = {},
            searchHint = stringResource(R.string.hint_search_query_actors),
            onSearchQueryChange = {},
            data = ActorSearch(
                personList = fakePersonsList(),
                isSearchingResults = false
            )
        )
    }
}

The test preview calls our custom preview function ensuring the UI is rendered using your actual app theme and data.

2.3 Generating Baseline Images

After setting up the test class, the next step is to generate reference images for each preview. These images serve as a baseline for future comparisons.

Gradle command to generate the reference images :

gradlew updateDebugScreenshotTest

After running this task, reference images are generated under:

app/src/debug/screenshotTest/reference/com/developersbreach/composeactors/screenshotTests/

2.4 Generating a test report

Next step is to run the validation task to capture a new screenshot and compare it with the baseline image.
For that we can run this command :

gradlew validateDebugScreenshotTest

This validation task will generate an HTML report located at {module}/build/reports/screenshotTest/preview/{variant}/index.html. This report will help you track any visual changes between the current and reference images.

This image displays the XML report that provides detailed results of the screenshot test, showing the test case execution status and time taken.

This image shows the HTML report highlighting visual changes between the current and reference images:

Voila!
You’ve successfully created and validated your first screenshot test in Jetpack Compose. Now, your UI is more reliable, and visual bugs are caught before your users even notice!

3. Running Screenshot Tests in CI/CD

While we run the tests locally we use gradle command as below

./gradlew validateDebugScreenshotTest

Say you already have other Gradle tasks configured in your GitHub Actions workflow, you can easily join the screenshot testing task with your existing commands.
If your workflow script already runs unit tests and instrumented tests, simply append the new task to your Gradle command.

script: |
  ./gradlew testDebugUnitTest validateDebugScreenshotTest connectedDebugAndroidTest --stacktrace

• testDebugUnitTest runs your unit tests.
• validateDebugScreenshotTest generates and compares Compose screenshot images against your reference images.
• connectedDebugAndroidTest runs your instrumented tests on the emulator.

You can find the whole workflow file here, the key script would look as below, where it creates a virtual android device with mentioned attributes for device configuration and runs the mentioned gradle commands.

- name: Create Android Virtual Device and run all tests
  uses: reactivecircus/android-emulator-runner@v2
  id: testing
  with:
    api-level: 29
    target: google_apis
    arch: x86_64
    force-avd-creation: true
    emulator-options: -wipe-data -no-snapshot-save -no-window -gpu swiftshader_indirect -noaudio -no-boot-anim -camera-back none
    disable-animations: false
    script: |
      ./gradlew testDebugUnitTest validateDebugScreenshotTest connectedDebugAndroidTest --stacktrace

4. Internals and failing tests

4.1 How Screenshot Testing Works Internally

When a screenshot test runs, Compose renders UI from @Preview functions into a bitmap image. This snapshot represents the current visual state of your component. If the images don’t match, the test fails and produces a report to help you compare and find the differences.

4.2 When Will a Screenshot Test Fail?

Have a look at both of the images, Reference and Actual image you will notice there is a little bit more padding to the left side. Hence the test will fail because it checks pixel by pixel, which is referenced in Diff image.

The gradle execution task failure will point us at the problem.

A small shift can cause the diff to exceed the threshold, resulting in a test failure that alerts you to the visual inconsistency.

• Unintended UI Changes:
  If a layout alteration, like a shifted button or changed padding causes the rendered image to differ significantly from the baseline.
• Color or Font Variations:
  Changes in colors, font sizes, or themes that are not part of the intended design update will result in a failing diff.
• Dynamic Content:
  UI elements that display dynamic data may cause inconsistent screenshots. Stabilizing such content is critical to avoid false failures. So keep it short to static content and have your previews with mock data.

Conclusion :

• UI Verification:
  Screenshot tests capture the entire rendered UI and compare it pixel by pixel.
• Automatic Regression Detection:
  They flag any unintended visual changes whether due to layout shifts, color changes, or environmental differences.
• Prevents Regressions:
  As your app evolves, changes made to one part of the UI can unintentionally affect others. Screenshot testing prevents such regressions by continuously comparing new builds with previous versions.
• Fast and Reliable:
  Running on the host JVM, these tests provide rapid feedback, ensuring your UI remains consistent across updates.
• Integration:
  By reusing existing Compose previews, you maintain a single source of truth for your UI design.

References :

Compose Preview Screenshot Testing
Project with screenshots tests implementation

Author

Shreyas Deshmukh

I’m a CSE student and an Android developer specializing in Kotlin and Jetpack Compose. I love building intuitive mobile applications and exploring innovative solutions to create impactful user experiences.

Explore articles

Reviewer & Editor

Raj

Hi ! I’m a Software Engineer at MedKitDoc & Technical writer.
I lead organization Developers Breach focusing on contributing to Open Source and Student Projects.

Explore articles