Skip to content

Port BasicAddMissingReference.InvokeSomeFixesInVisualBasicThenVerifyReferences to the new test framework#57626

Merged
sharwell merged 6 commits intodotnet:mainfrom
sharwell:vs-extension-testing
Nov 12, 2021
Merged

Port BasicAddMissingReference.InvokeSomeFixesInVisualBasicThenVerifyReferences to the new test framework#57626
sharwell merged 6 commits intodotnet:mainfrom
sharwell:vs-extension-testing

Conversation

@sharwell
Copy link
Copy Markdown
Contributor

@sharwell sharwell commented Nov 8, 2021

Roslyn's integration tests use a custom harness, where test code executes in a separate process (testhost.x86.exe or similar) and use .NET Remoting to invoke methods on objects within a Visual Studio instance. This pull request creates a second integration test project using the new test harness from microsoft/vs-extension-testing. In addition to moving maintenance of the test harness infrastructure to a common repository, this change significantly reduces integration test reliance on .NET Remoting in hopes of improving test performance and reliability by executing test code directly within the Visual Studio instance. It also standardizes diagnostics collection (screenshots, Windows event logs, the activity log, and IDE state at point of failure).

This change ports one notoriously flaky test from Roslyn's harness to the new harness, along with its required supporting methods. The new code differs primarily in the following ways:

  • Where possible, non-DTE Visual Studio APIs are preferred to DTE APIs
  • Where possible, operations are written as asynchronous (this especially includes the test itself, since the test entry point is now the main thread of a Visual Studio instance)
  • Since tests run directly within Visual Studio, there is no need to separate in-proc and out-of-proc test objects

The goal of the new test project in the current limited form is twofold:

  1. Evaluate the reliability of the test when run under the new harness
  2. In the event of failure, compare our ability to diagnose the failure in the new harness with information provided by the old harness

@ghost ghost added the Area-IDE label Nov 8, 2021
@sharwell sharwell force-pushed the vs-extension-testing branch from 734f5bf to b2e1d2f Compare November 9, 2021 00:38
@sharwell sharwell force-pushed the vs-extension-testing branch from c66ecb5 to 8bb0dcf Compare November 10, 2021 17:19
@sharwell sharwell marked this pull request as ready for review November 10, 2021 20:07
@sharwell sharwell requested review from a team as code owners November 10, 2021 20:07
@sharwell sharwell requested a review from a team November 10, 2021 20:07
Comment thread src/VisualStudio/IntegrationTest/New.IntegrationTests/AbstractEditorTest.cs Outdated
await JoinableTaskFactory.SwitchToMainThreadAsync(cancellationToken);

var filePath = await GetAbsolutePathForProjectRelativeFilePathAsync(projectName, relativeFilePath, cancellationToken);
VsShellUtilities.OpenDocument(ServiceProvider.GlobalProvider, filePath, VSConstants.LOGVIEWID.Code_guid, out _, out _, out _, out var view);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc there is an async document open service now - https://devdiv.visualstudio.com/DevDiv/_git/VS.RPC.Contracts?path=%2Fsrc%2FMicrosoft.VisualStudio.RpcContracts%2FOpenDocument%2FIOpenDocumentService.cs&_a=contents&version=GBmain

It might more reliably set the focus and there are options to activate it, set selection, set visible ranges, etc.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea how one would use this. @olegtk ?

@sharwell sharwell merged commit f948cdc into dotnet:main Nov 12, 2021
@sharwell sharwell deleted the vs-extension-testing branch November 12, 2021 19:22
@ghost ghost added this to the Next milestone Nov 12, 2021
@allisonchou allisonchou modified the milestones: Next, 17.1.P2 Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants