Skip to content

Analyzer testing restructure #2077

@tlakollo

Description

@tlakollo

Given the increase in the number of tests for the different analyzers, we have now repeated several of the tests to cover different attribute usages. In general, we have two approaches to reduce the number of repeated tests:

  • Share tests between the linker and the analyzer. Given that ideally the linker and the analyzer should produce the same diagnostics sharing the tests between the linker and analyzer makes sense. The problem is that we can only share tests for things that are tested in the linker repo, this means that we cannot test RAF as part of the linker tests since the linker doesn't conduct a global analysis of this attribute.
  • Share tests between analyzers. The RAF and RUC analyzer tests are similar and therefore we can use a base class to share the test between them.

The proposal is an idea originally by @mateoatr to modify the ExpectedWarning attribute to be able to differentiate between a warning that is generated by the linker, the analyzer, or both. This way we could have analyzer validation inside linker tests and cover analyzer scenarios by reusing tests in the linker. Consider the following example:

[ExpectedWarning ("IL2026", "Message for --RequiresWithMessageOnly--.")]
// ExpectedTypes includes something like Analyzer, Linker, and All values, All is the default
[ExpectedWarning ("IL3002", ExpectedBy = ExpectedTypes.Analyzer]
static void TestRequiresWithMessageOnlyOnMethod ()
{
	RequiresWithMessageOnly ();
}

[RequiresUnreferencedCode ("Message for --RequiresWithMessageOnly--")]
[RequiresAssemblyFiles]
static void RequiresWithMessageOnly () {}

The result checker in the linker will check if the ExpectedTypes.Linker is active, otherwise; it would disregard the warning. We could do something similar for the analyzer testing infrastructure to test only what is marked with ExpectedTypes.Analyzer. Explaining the code scenario the warning IL2026 would be expected for both the analyzer and the linker, meaning that both testing infrastructures would check for the warning being generated. In the case of IL3002 the result checker in linker will bypass the diagnostic but the test infrastructure of the analyzer will check for the diagnostic.

This would simplify the testing since there would be only 1 testing file in the linker for requirescapability that tests RAF and RUC on both linker and analyzer.

There would be still analyzer testing since the CodeFixers cannot be tested in the linker infrastructure.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions