-
-
Notifications
You must be signed in to change notification settings - Fork 837
Closed
Description
A test which is marked as explicit will normally not be run with a "run all" gesture, but will be run when asked for explicitly.
- For the SDK, we will need some metadata to indicate explicit tests (they should have a message, much like skipped tests do today) as well as an object model object which indicates a test which was skipped because of being marked as explicit (to differentiate from tests which are skipped because of being marked as skipped). The APIs to "run all" will not run explicit tests, but any API which takes a test case list will run explicit tests if they're requested. This means that runners should be careful to segregate explicit tests when filtering (give help here when possible with the existing filter classes).
- For theories, the theory data model in v3 has been extended to allow skipping individual data rows. The theory data model should be extended to also allow the data row to be marked as explicit.
- For console runners, the default run all behavior will not run explicit tests (and we should add a reporting of explicit tests). A new option should be added (
-explicit?) which will include explicit tests. - For the MSBuild runner, the default run all behavior will not run explicit tests (and we should add a reporting of explicit tests). There is no equivalent to
-methodor-classin the MSBuild runner. A new option should be added (Explicit=?) which will include explicit tests. - For the VSTest runner, the "run all" API should skip explicit tests and report explicit tests with an outcome of
TestOutcome.None; the "run some" API should run any explicit test that's in the list.
I think there's some design work TBD yet. Some questions:
- Are the only options for
-explicitsomething liketrue,false, etc.? Should there, for example, be a way to run only explicit tests (f.e.,-explicit only)? - Should
-classrun explicit tests in the given class? How is this impacted by-explicit? - Should
-methodrun an explicit test method? How is this impacted by-explicit? What about when the method is a theory, but only some of the data has been marked as explicit? - Test filtering expressions should be part of the consideration for explicit tests (and probably match the features available via
-explicit).
Discussion: #2483
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels