Skip to content

Multi package workspace - can't find all tests to execute #3712

@up-doc

Description

@up-doc

Describe the bug
A bug in the test finder and therefore the overall number of tests ran.

When in a multi package workspace, if the package folders are all at the same directory level, then the test finder discovers all of the tests and runs them. See packages_all_on_one_level attached zip. This finds all of the workspace's 702 tests and runs them correctly.

When the packages are at another folder level deep. The test finder doesn't find all the tests. Having a main app folder and a packages, shared or features folder for other packages is a common workspace structure for mono repo applications. So having this issue is a blocker for this extension.

When you open the folder for packages_one_level_deep attached zip, you run and see that it only executes 312 tests. It misses tests that in subfolders of the shared packages.

I have also noticed an error, that if you run Test: run all tests before tapping on the test tab, it fails with an error

no_found_tests

one_level_deep
all_same_level

Final issue I'd like to raise is the naming of the test suites as it finds tests within a package. The test suites appear to be named after the first test that the test finder finds in the package, rather than any meaningful package scoped name. See image below.

Screenshot 2021-12-06 at 21 06 53

To Reproduce
Steps to reproduce the behavior:

  1. Open packages_all_on_one_level and go to test navigator and run all tests
  2. See that there are the correct 702 Tests
  3. Open packages_one_level_deep and go to test navigator and run all tests
  4. See that there are the incorrect 312 Tests
  5. Look in found tests and can see that it is missing tests located under subfolder1, anotherSubfolder3 and other subfolders containing 26 tests each.

Expected behavior
Finds all the tests in any folder, executes all and gives correct number of passed/failed tests.

Also allows for folders to be skipped would be nice to. As sometimes we have code in our workspaces that isn't ours :)

Screenshots
See screenshots above

Reproduced with:
packages_all_on_one_level.zip

packages_one_level_deep.zip

Versions (please complete the following information):

  • VS Code version: 1.62.3
  • Dart extension version: v3.29.0
  • Dart/Flutter SDK version: Flutter 2.2.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    in testingRelates to test execution of Dart/Flutter tests for end usersis bug

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions