When developing applications, it's very common for a small change in the code to end up moving a button a few pixels or altering a color without us noticing. These situations, known as visual regressions, often go undetected in traditional unit tests because, even though the code works, the Visual appearance degradesaffecting the end-user experience.
To prevent the product from reaching production with aesthetic defects, the following has emerged: Screenshot TestingBasically, it involves comparing a current screenshot with a previously approved reference image; if there is a difference, the system alerts us so we can correct the error or update the reference if the change was intentional.
Understanding visual regression and its challenges
A visual regression occurs when there is an unwanted deviation in rendering, such as a skipping spacing or a component that doesn't fold correctly on mobile devices. The problem is that the DOM or view hierarchy may remain identical, but the visual result is incorrect. To combat this, we need deterministic capture and a diff system capable of distinguish real errors of irrelevant variations, such as anti-aliasing.
There are two main approaches to running these tests on Android. On the one hand, there are those that run directly on the device, which are very accurate but slow. On the other hand, we have those that are based on... previewswhich are much faster and easier to run as they do not require an emulator or physical phone, although sometimes they may not reflect 100% the reality of the device.
The Screenshot Testing tool in Compose
Google has introduced an experimental feature called Compose Preview Screenshot Testing. This tool allows you to convert Compose previews into automated testsThe flow is simple: a preview is selected, a base image is generated, and then it is validated that any subsequent changes do not break the defined aesthetic.
Thanks to this integration, we can use parameters from the @Preview annotation, such as the font scaling or night modeto ensure that the interface is consistent across multiple configurations without having to test each one manually. Furthermore, it allows for modular testing through a new set of origins called screenshotTest.
Technical configuration and requirements
To get this up and running, it's essential to have the correct versions of the environment. If you're looking for full integration with the IDE, you'll need Android Studio Canary 3 Feature Drop Canary 4 or higher, along with AGP version 9.0 and Kotlin 2.2.10. If you prefer to use only Gradle tasks, the requirements are more flexible, allowing versions such as AGP 8.5.0 and Kotlin 1.9.20.
The setup process begins by activating the experimental property android.experimental.enableScreenshotTest=true in the File gradle.propertiesSubsequently, the same brand must be enabled within the android block in the build file and the plugin added. com.android.compose.screenshot.
Don't forget to add the necessary dependencies, specifically the screenshot-validation-api and the bookstore of ui-tooling, since they are the engine that allows the capture and validation of interfaces in the test environment.
Implementation of tests and generation of images
To designate which components we want to test, we must use the annotation @PreviewTest Regarding composable functions, these must reside in the path src/screenshotTest/kotlinOnce created, the next step is to generate the reference imageswhich will serve as the "absolute truth" for future comparisons.
In the IDE, this is as easy as clicking the margin icon and selecting the option to add or update images. If you prefer the terminal, you can run the command ./gradlew updateDebugScreenshotTestThese images are saved locally, and it is recommended to add the references folder to the .gitignore to avoid overloading the Git repository with binary files.
Validation of results and failure analysis
Once we have the references, we run the validation using ./gradlew validateDebugScreenshotTestIf something has changed, the system will generate a HTML report detailed. Android Studio offers a built-in difference viewer where we can compare the reference image, the current image, and the difference image (diff) in parallel, using zoom and pan tools.
The report also provides useful metadata, such as percentage of match and the image dimensions. If the test fails due to an expected design change, we simply need to update the reference image. If it's a bug, we can go directly to the composable to fix the visual bug.
Broad perspective: From local to full automation
Beyond Jetpack Compose, frameworks like Selenium, Appium, and Playwright allow you to integrate visual regressions into websites and mobile apps. Tools like TestingBot make this easier by adding a single line of code. In corporate environments, the AI is being integrated to help detect error patterns that a human eye might miss, optimizing software quality.
The adoption of a robust baseline methodology, combined with services of cybersecurity and cloud deployment (like AWS or Azure), ensures that the application is not only aesthetically pleasing, but also secure and scalable. Ultimately, the goal is to minimize the costs of correcting errors later by detecting them in the early stages of development.