Read E2E logs and fail the job if no tests matched the filter#62222
Read E2E logs and fail the job if no tests matched the filter#62222ilonatommy merged 18 commits intodotnet:mainfrom
Conversation
|
The alternative way of running triggered the test process. We got an error informing about missing xunit.runner.console package: Locally we don't experience these problems using |
build.sh -test fails the same way dotnet test doesdotnet test failing to discover tests
|
The binlog of |
dotnet test failing to discover tests|
This PR's CI is supposed to fail, to show that the mechanism is working. After merging #62250 it should go green. |
Youssef1313
left a comment
There was a problem hiding this comment.
Or, migrate to xunit.v3+MTP where zero tests ran is error by default :)
This is an interesting point. Is #62250 doing that? Does "arcade's default" mean v3? |
|
Nope. My PR only uses But happy to help you with the migration. It was done for Aspire (dotnet/aspire#8403 and dotnet/aspire#8498), and in progress for winforms (dotnet/winforms#13540). We also did it for Uno Platform (https://platform.uno/blog/microsoft-contributes-to-uno-platform-dramatically-reducing-testing-time/), and many more internal (and external) repos. |
|
|
@lewing, what are your thoughts about xunit upgrade? Should we go for it right now or rather grep for now and log that as a future work? |
…n the total test number to make the CI fail.


Trying to avoid the situation we faced with #62223
Description
For a few days we were merging PRs that did not run through all the tests. If we had a mechanism that would check the test output for "No test matches the given testcase filter" and would fail the job, we would have caught it earlier.
I am aware that not only E2E tests were affected, template tests or generally, tests running on Helix also had this problem, I don't see an easy way how to use a similar mechanism there.