Allow addition of custom test runner #12509
Description
- This issue is blocking
- This issue is causing unreasonable pain
I'd like to add a custom test runner to run mutation tests (https://stryker-mutator.io/docs/stryker-net/introduction/). However, it appears an impossible task - the Arcade SDK expects all test runners to reside within:
Can we relax this condition and load a custom test runner from a target repository? E.g., something like this
diff --git a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets
index ffb58097..1c2bc8e9 100644
--- a/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets
+++ b/src/Microsoft.DotNet.Arcade.Sdk/tools/Tests.targets
@@ -16,7 +16,8 @@
<TestRunnerName Condition="'$(TestRunnerName)' == ''">XUnit</TestRunnerName>
<TestRunnerName Condition="'$(UsingToolXUnit)' == 'false'"></TestRunnerName>
- <TestRunnerTargets>$(MSBuildThisFileDirectory)$(TestRunnerName)\$(TestRunnerName).targets</TestRunnerTargets>
+ <TestRunnerTargets>$(RepositoryEngineeringDir)$(TestRunnerName)\$(TestRunnerName).targets</TestRunnerTargets>
+ <TestRunnerTargets Condition="!Exists($(TestRunnerTargets))">$(MSBuildThisFileDirectory)$(TestRunnerName)\$(TestRunnerName).targets</TestRunnerTargets>
</PropertyGroup>
<Target Name="ErrorForMissingTestRunner"
@@ -80,7 +81,7 @@
</Target>
<!-- Import specialized targets files of supported test runners -->
- <Import Project="$(MSBuildThisFileDirectory)$(TestRunnerName)\$(TestRunnerName).targets" Condition="'$(IsTestProject)' == 'true' and '$(TestRunnerName)' != '' and Exists('$(MSBuildThisFileDirectory)$(TestRunnerName)\$(TestRunnerName).targets')"/>
+ <Import Project="$(TestRunnerTargets)" Condition="'$(IsTestProject)' == 'true' and '$(TestRunnerName)' != '' and Exists('$(TestRunnerTargets)')"/>
<!-- Allow for repo specific Test targets such as rerunning tests -->
<Import Project="$(RepositoryEngineeringDir)Tests.targets" Condition="Exists('$(RepositoryEngineeringDir)Tests.targets')" />This will allow me (and anyone for that matter) create and plug a custom test runner, or override and customise an existing runner implementation (this is probably a far-fetched usecase, but why not?).
For example, I've created a /eng/StrykerNET folder with StrykerNET.targets, which implements RunTests target:
<Target Name="RunTests"
Outputs="%(TestToRun.ResultsStdOutPath)"
Condition="'@(TestToRun)' != ''">
<!-- -->
</Target>In Directory.Build.props I set this property and everything just works:
<PropertyGroup>
<TestRunnerName>StrykerNET</TestRunnerName>
</PropertyGroup>
Release Note Category
- Feature changes/additions
- Bug fixes
- Internal Infrastructure Improvements