Add android_hardware_smoke_test integration tests#187130
Conversation
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces the android_hardware_smoke_test integration test suite, which supports both standalone on-device testing for OEMs and host-driven CI testing. It registers Vulkan and OpenGLES test shards, sets up custom Gradle tasks to embed test screenshots into HTML reports, and implements the necessary Dart and Android native harnesses. The review feedback identifies a critical regex mismatch that silently breaks backend selection, warns of binary corruption on Windows hosts when using adb shell instead of adb exec-out, recommends robust error handling and null safety in the golden comparison logic to prevent test hangs, and suggests minor style guide and dependency pinning improvements.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new integration and compatibility smoke test suite, android_hardware_smoke_test, designed to verify visual rendering correctness on Android hardware using both an on-device instrumented mode and a host-driven driver mode. The feedback suggests refining the regular expression used to match the ImpellerBackend metadata tag to prevent greedy matching, using JUnit 4 assertions consistently in FlutterActivityTest.java, increasing test timeouts to avoid CI flakiness, and checking the exit code of the adb exec-out process in the Gradle task to handle potential transfer failures gracefully.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces the android_hardware_smoke_test integration test suite, designed to verify visual rendering correctness on Android hardware using both a self-contained instrumented mode and a host-driven driver mode. The changes include the test suite setup, native Android activity configurations, and a custom Gradle task to embed result screenshots into the HTML reports. Feedback is provided on improving the robustness of the regular expression used to match the Impeller backend, resolving potential thread/resource leaks in the Java test executor, adhering to Google Java Style Guide spacing rules for binary operators, and improving error handling when loading golden assets or casting render objects.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new integration and compatibility smoke test suite, android_hardware_smoke_test, to verify visual rendering correctness on Android hardware using a dual-mode architecture (instrumented and driver modes). Key feedback includes making a regular expression in the suite runner non-greedy, refactoring golden image comparison helpers to return a Future instead of passing a Completer to avoid potential double-completion errors, explicitly specifying offset and length when converting ByteData to Uint8List to prevent reading incorrect buffer data, and adding a null check for activity.messageChannel in the native test harness to prevent a potential NullPointerException.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces the android_hardware_smoke_test integration test suite to verify visual rendering correctness on Android hardware. Feedback on the changes includes addressing a critical Dart syntax error where Swift-style implicit member syntax is used instead of ColorScheme.fromSeed. Additionally, suggestions are provided to make the regular expression in the test runner more robust, simplify redundant try-catch blocks in goldens.dart, and improve the error handling in the Gradle build script when querying sandbox files via ADB.
…nding and explain its purpose
|
Verified CI using |
Create a new suite of integration tests which can run either in CI or directly on an android device as APKs. See the readme for more details on how it works.
This begins to address #182123
We'll expand coverage in future PRs after we verify that this runs well in CI.
Pre-launch Checklist
///).