Skip to content

How to debug integration tests #88756

Description

@mleonhard

Use case

I'm developing an integration test for a mobile app. When the test fails, the test log says "Failed to stop app" and the app shows a black screen. I need some way to examine the state of the app and find out why it failed.

I am not asking to run the test in a debugger. I just want to fix my broken test. The Flutter tooling makes this very difficult.

Adding a dart:io:sleep() or await Future.delayed() leaves the app's UI frozen and not scrollable.

So far the best I can do is add print statements and take a video of the app in Apple Simulator and step through it frame-by-frame. This is a slow and frustrating way to fix tests.

Proposal

Provide some way to examine the state of the app when an integration test fails.

Ideas:

  • Support flutter_driver and make it usable. One can comment out the call to FlutterDriver.close() and the app will remain live and usable after the test stops.
  • Leave the app open and usable, not frozen. Do not show a black screen.
  • Explain how to run a test in the Android Studio debugger. Let me set a breakpoint before the failing step. Then I can examine the app's widget tree, the app's data, and the test's data.
  • Automatically take a screenshot on test failure. I'm not sure how to make this work since integration_test module runs on the target device and has no access to the test runner's filesystem to save the file.
  • Make a video of the entire test. Delete the video on success. Leave it on failure. This could end up using a lot of disk space.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projecta: tests"flutter test", flutter_test, or one of our testsc: new featureNothing broken; request for a new capabilityc: proposalA detailed proposal for a change to Flutterf: integration_testThe flutter/packages/integration_test pluginframeworkflutter/packages/flutter repository. See also f: labels.t: flutter driver"flutter driver", flutter_drive, or a driver testteam-frameworkOwned by Framework teamtriaged-frameworkTriaged by Framework team

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions