Skip to content

moved waiter from diagnostics.dll to features.dll where all interface…#24120

Merged
heejaechang merged 20 commits intodotnet:dev15.6.xfrom
heejaechang:mergeWaiter
Jan 17, 2018
Merged

moved waiter from diagnostics.dll to features.dll where all interface…#24120
heejaechang merged 20 commits intodotnet:dev15.6.xfrom
heejaechang:mergeWaiter

Conversation

@heejaechang
Copy link
Contributor

@heejaechang heejaechang commented Jan 9, 2018

…s are defined.

Customer scenario

This doesn't affect customer experience. this is purely for testing.

Bugs this fixes

https://devdiv.visualstudio.com/DevDiv/_workitems/edit/549239

Workarounds, if any

install our Roslyn.Test.Setup.vsix to machines that want to test our asynchronous features.

Risk

This only affects Testing, so there shouldn't be user facing risk.

Performance impact

There should be no perf impact due to this change to users. only affects testing.

Is this a regression from a previous update?

N/A

Root cause analysis

As all other Roslyn design, we separated out test related component from product component, and used vsix and MEF to inject those when it is needed (test env). it gave us clean separation. but gave us yet one more thing to setup when running test.

it was fine for our own test env, but when it meets with other components in VS and when other team needed to run some of their test with our test hook, it became yet one more point that can fail for test setup.

so, we now put those hooks in Roslyn dlls itself which can be enabled by env variable.

How was the bug found?

during testing.

@heejaechang heejaechang requested a review from a team as a code owner January 9, 2018 11:23
@heejaechang heejaechang added the PR For Personal Review Only The PR doesn’t require anyone other than the developer to review it. label Jan 9, 2018
namespace Roslyn.Hosting.Diagnostics.Waiters
{
internal abstract class EditorAdornmentWaiter : AsynchronousOperationListener
internal abstract class EditorAdornmentWaiter
Copy link
Contributor Author

@heejaechang heejaechang Jan 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left these files since I wasn't sure what this is trying to accomplish. since it continues after task being completed. the task itself shouldn't rely on UI thread being ran. so I wasn't sure what this Dispatcher is trying to do. if there is some work scheduled to UI thread, the original task should wait for it so, the chained dispatcher doesn't have any impact?

@heejaechang
Copy link
Contributor Author

heejaechang commented Jan 11, 2018

@dotnet/roslyn-infrastructure turns out the failure in integration test has nothing to do with my change. it was just a bug in integration test. I am not sure how it was ever worked...?
...
never mind, found out why it used to work.

@heejaechang
Copy link
Contributor Author

@jasonmalinowski @dotnet/roslyn-ide @dotnet/roslyn-analysis can you take a look?

I simplified our listener/waiter thing. and put everything basically in one central end point (AsynchronousOperationListenerProvider).

it no longer uses MEF for each listener. only MEF is the provider itself. and it is MEF because I am using MEF for isolation in unit test.

all other changes are just replacing one to the other. actual change is in the provider.

@heejaechang
Copy link
Contributor Author

@brettfo so, you added a way to call our Waiter in apex code? @ivanbasov can you point me how I can update Apex code?

{
get
{
if (!s_enabled.HasValue)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't use lock for this and the one below. these are not expected to called multiple time. but only once for 1 VS session.

@brettfo
Copy link
Member

brettfo commented Jan 11, 2018

@heejaechang: The change to consume this waiter in Apex is already in the internal code. I'll email you the details.

@heejaechang heejaechang removed the PR For Personal Review Only The PR doesn’t require anyone other than the developer to review it. label Jan 11, 2018
Copy link
Member

@jasonmalinowski jasonmalinowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My biggest complaint is we didn't do this sooner! Thanks for cleaning this one up.

A few nitpick questions, and I'd like a few bits of strangeness around interfaces/concrete types fixed up where we have some casts that make you go "hmm?".

.GetExports<IAsynchronousOperationListener, FeatureMetadata>()
.First(l => l.Metadata.FeatureName == FeatureAttribute.Workspace).Value as IAsynchronousOperationWaiter;
.GetExportedValue<IAsynchronousOperationListenerProvider>()
.GetListener(FeatureAttribute.Workspace) as IAsynchronousOperationWaiter;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing the "as" is no longer necessary here? (and I'm not sure why we'd ever use as since it'll null ref if it's wrong...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep. it wasn't need to. I was just lazy :) changing to as ... was quicker than move to front and put (...) :) but sure, I can change it to cast.

{
var waiters = Workspace.ExportProvider.GetExportedValues<IAsynchronousOperationWaiter>();
await waiters.WaitAllAsync();
var provider = Workspace.ExportProvider.GetExportedValue<IAsynchronousOperationListenerProvider>() as AsynchronousOperationListenerProvider;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fishy here to be grabbing by the interface and then an "as" cast when (a) it should be a hard cast and (b) why can't we just export the concrete type directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure. I can expose concrete type as MFE as well. I believe same type expose 2 MEF types, it still uses same instance right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetWaiter doesn't exist in interface since waiter is not supposed to be used in product. and the interface is for the product code.

internal WorkspaceWaiter()
{
}
return (AsynchronousOperationListenerProvider)provider.GetExportedValue<IAsynchronousOperationListenerProvider>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the concrete type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

waiter only exist in concrete type. interface only provide a way to get listener.

SpecializedCollections.SingletonEnumerable(
New Lazy(Of IAsynchronousOperationListener, FeatureMetadata)(
Function() waiter, New FeatureMetadata(New Dictionary(Of String, Object)() From {{"FeatureName", FeatureAttribute.Classification}}))))
listenerProvider)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well that's a lot simpler. :-)


Dim waiters = workspace.ExportProvider.GetExportedValues(Of IAsynchronousOperationWaiter)
Await waiters.WaitAllAsync()
Dim provider = DirectCast(workspace.ExportProvider.GetExportedValue(Of IAsynchronousOperationListenerProvider), AsynchronousOperationListenerProvider)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As earlier comment: odd to have both interface and concrete type here...


private IEnumerable<IAsynchronousOperationWaiter> GetCandidateWaiters(string[] featureNames)
{
if (featureNames == null || featureNames.Length == 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I'd consider null OK but an empty list to be an error...

Copy link
Contributor Author

@heejaechang heejaechang Jan 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

method(params string[] xxx) gives empty array, so need to check empty array. this particular method doesn't use params, but most of caller to this method does and i didn't want to put if (xx.length == 0)? return null to all those caller :)

namespace Roslyn.Hosting.Diagnostics.Waiters
{
[Export, Shared]
public class TestingOnly_WaitingService
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we even need this type anymore? It seems like the other type does all the same stuff?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is public. this is for ones that doesn't have access to roslyn, but want to use waiter as long as they can somehow get to this Roslyn.xxx.Disagnostics.dll

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, our integration test uses it. so.. I left it.

{
internal abstract class EditorAdornmentWaiter : AsynchronousOperationListener
{
public override Task CreateWaitTask()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm horrified this was virtual and had a special case like this. Do we still need this logic? (I hope not...?)

Copy link
Contributor Author

@heejaechang heejaechang Jan 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way it does was incorrect I believe. since it basically wanted to call DoEvent for other waiter (not this waiter). now I explicitly call Doevent while waiting rather than some random waiter calling it hoping that is enough for other waiter stuck waiting for UI context to proceed.

@@ -1,16 +0,0 @@
' Copyright (c) Microsoft. All Rights Reserved. Licensed under the Apache License, Version 2.0. See License.txt in the project root for license information.

#If False Then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!


private VisualStudioWorkspace_InProc()
{
// we need to enable waiting service before we create workspace
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you'd want to move this to somewhere earlier when we are actually setting up the cross-process communication. This might be too late at this point.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed this is early enough. but if you can tell me where is the first point where we start the inproc process, I can put it there as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, I didn't realize all these get force-connected during us bringing up the instance. This will work (albeit in ugly ways but that's not your fault.)

@jasonmalinowski
Copy link
Member

@heejaechang Do we need/want this in dev15.6.x so we can land the fix into the VS release branches sooner? I presume they'd want this sooner.

{
var signal = new ManualResetEventSlim();
var listener = new Listener();
var listener = new AsynchronousOperationListener("test", tracking: false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these all tracking: false? I don't think it's bad, it just seems odd to be called out directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, this code is before I made them default parameter. let me remove those.

@heejaechang
Copy link
Contributor Author

retest windows_release_vs-integration_prtest please

@heejaechang
Copy link
Contributor Author

ping?

Copy link
Member

@brettfo brettfo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I was able to follow, :shipit:

@heejaechang
Copy link
Contributor Author

@heejaechang
Copy link
Contributor Author

@jinujoseph @Pilchie can I get approval for ask mode?

@heejaechang
Copy link
Contributor Author

@jasonmalinowski can you take a look when you have time. I addressed your feedbacks.

@heejaechang heejaechang changed the base branch from master to dev15.6.x January 12, 2018 23:36
@Pilchie
Copy link
Member

Pilchie commented Jan 16, 2018

Approved - thanks @heejaechang

@heejaechang
Copy link
Contributor Author

@jasonmalinowski so are you okay with the change? I addressed your PR feedbacks.

@heejaechang heejaechang merged commit 823d973 into dotnet:dev15.6.x Jan 17, 2018
jasonmalinowski added a commit to jasonmalinowski/roslyn that referenced this pull request Jan 25, 2018
heejaechang pushed a commit that referenced this pull request Jan 26, 2018
* Add UseCultureAttribute to help with culture-dependent unit tests

* Disable NuGet package restore in Visual Studio for Roslyn.sln

* Add named argument for literal

* Fix behavior for NET46 and NETCOREAPP2_0

* Revert "moved waiter from diagnostics.dll to features.dll where all interface… (#24120)"

This reverts commit 823d973.

* Add references to System.IO.Pipes.AccessControl (#24457)

When adding the reference to System.IO.Pipes.AccessControl for the
compiler server to use on CoreCLR, I unified the pathway for the desktop
and CoreCLR server access control code. This means that
System.IO.Pipes.AccessControl needed to be added as a dependent DLL for
desktop too, but I forgot to do that. This change adds
System.IO.Pipes.AccessControl as a dependent DLL in all the places where
the build task is deployed.
heejaechang added a commit to heejaechang/roslyn that referenced this pull request Jan 29, 2018
heejaechang added a commit that referenced this pull request Jan 31, 2018
…s are defined. (#24512)

* put listener change back in (#24120)

* leave old types in legacy folder until partner teams move to new interface

* added legacy waiter to support partner teams
heejaechang pushed a commit that referenced this pull request Feb 2, 2018
* Remove duplicate lock DocumentState.s_syntaxTreeToIdMapLock

This lock is only being used to protect access to an instance which contains
internal synchronization.

* Better handle surrounding directives when inlining a local variable.

* Add tests.

* Share code between VB and C#.

* Reduce allocations in UnboundLambda

Fixes #23463

* Restore ReturnInferenceCacheKey as the key for _returnInferenceCache

* Update code to more closely follow patterns of the original code

* Cleanup from code review

* basic fix for intellisense in Immediate window

* better comments and cleanup

* Add basic integration tests

* cleanup inproc Immediate window integration test helper

* fix incorrect comment

* address PR feedback

* create Immediate window on ImmediateWindow_InProc.GetText()

* Verify MSBuild version in Developer CMD prompt

Roslyn is designed to have the simplest possible contribution story:
clone then build. Every pre-req needed is either located on the machine
or bootstrapped via NuGet. All the way down to using an xcopy MSBuild if
needed.

The one case which causes a problem is the VS command prompt. In this
case MSBuild is pre-installed on the machine and may or may not be
suitable for building Roslyn.

Previously when building from a VS command prompt we just used whatever
MSBuild was provided. The assumption being a developer command prompt
was an explicit statement of whath MSBuild you wanted to use. Based on
all of our customer reports though this does not seem to be the
assumption that consumers of our repo have. The build gave them no
explicit errors about the provided toolset and hence when the build
failed they assigned flakiness to our repo.

Going forward we are applying the same version validation to MSBuild
when provided via a developer command prompt. If it doesn't match we
will refuse to build asking the user to upgrade VS or build from a
normal command prompt.

* Remove unneeded debugging line

* Comment about pre-release

* Added minimum version

* Add Omit If Default style option

* Add space to be like test without the omit

* Add/Remove without needing a property

* Reformat

* PR feedback

* Fix VB diagnostic based on feedback

* Handle case of NotApplicable modifier and field declaration list

* Fix tests

* PR feedback

* PR feedback

* PreviewCodeAction was overriding ComputeOperations but returning a post-processed operation from original action. This results in another PostProcess being called on the codeaction. If postprocess was overriden in originalaction that'll be ignored the second time (#23920)

* Support negative null-check when we are suggesting to inline type checks

Fixes #21097
Fixes #24286

* fix a case where persistent storage registration fails and some clean… (#24458)

* fix a case where persistent storage registration fails and some clean up code around it.

* added readonly

* address PR feedback

* removed comments no longer relevant

* renamed lock name

* moved waiter from diagnostics.dll to features.dll where all interfaces are defined. (#24512)

* put listener change back in (#24120)

* leave old types in legacy folder until partner teams move to new interface

* added legacy waiter to support partner teams

* Remove methods indirecting access to _metadataFileNameToConvertedProjectReference

This field is documented as being written and read from any thread,
but in practice all uses are guarded by an AssertIsForeground(). Thus
we can get rid of the helper methods that are trying to "help" by
locking before accessing the fields, making it really hard to track all
the real uses of it.

* Make method static that doesn't need state

* add a comment to address PR feedback

* Fix up tests of P2P to metadata reference conversion

It turns out we had some tests, but the tests were disabled. This was
because the tests weren't working properly anyways: they were calling
into UpdateProjectBinPath which only updated some (but not all) of
the project state. That was an internal helper method that shouldn't
be used by tests. Updating the tests to use
SetBinOutputPathAndRelatedData works better.

* Delete debug-only reference validation

This was some legacy code that tried to verify that the references
we have from the project system match up to what DTE and other sources
say. This was debug-only, and the actual asserts were commented out.
This is deadweight at this point, so delete it.

* added and cleaned up logs around build and live diagnostics. (#24551)

also added RoslynActivityLogger that can be enabled through project-system-tool

* Avoid closure allocations on the BindSyntaxTreeToId fast path

* CS1628 error text mentions in parameters; fixes #24584

* Small cleanup of completion logic.

* Move to xunit.console for CoreClr tests

Previously we were using xunit.console for desktop tests and dotnet-xunit for our
CoreClr tests. This change unifies us on top of xunit.console (now that it has a
netcoreapp2.0 version available).

* Move unix builds to xunit.runner.console as well

* Get actual directory name, not file

* Fix dir name issue

* fixed build break
heejaechang pushed a commit that referenced this pull request Feb 7, 2018
* Remove duplicate lock DocumentState.s_syntaxTreeToIdMapLock

This lock is only being used to protect access to an instance which contains
internal synchronization.

* Better handle surrounding directives when inlining a local variable.

* Add tests.

* Share code between VB and C#.

* Reduce allocations in UnboundLambda

Fixes #23463

* Restore ReturnInferenceCacheKey as the key for _returnInferenceCache

* Update code to more closely follow patterns of the original code

* Cleanup from code review

* Verify MSBuild version in Developer CMD prompt

Roslyn is designed to have the simplest possible contribution story:
clone then build. Every pre-req needed is either located on the machine
or bootstrapped via NuGet. All the way down to using an xcopy MSBuild if
needed.

The one case which causes a problem is the VS command prompt. In this
case MSBuild is pre-installed on the machine and may or may not be
suitable for building Roslyn.

Previously when building from a VS command prompt we just used whatever
MSBuild was provided. The assumption being a developer command prompt
was an explicit statement of whath MSBuild you wanted to use. Based on
all of our customer reports though this does not seem to be the
assumption that consumers of our repo have. The build gave them no
explicit errors about the provided toolset and hence when the build
failed they assigned flakiness to our repo.

Going forward we are applying the same version validation to MSBuild
when provided via a developer command prompt. If it doesn't match we
will refuse to build asking the user to upgrade VS or build from a
normal command prompt.

* Remove unneeded debugging line

* Comment about pre-release

* Added minimum version

* Add Omit If Default style option

* Add space to be like test without the omit

* Add/Remove without needing a property

* Reformat

* PR feedback

* Fix VB diagnostic based on feedback

* Handle case of NotApplicable modifier and field declaration list

* Fix tests

* PR feedback

* PR feedback

* Support negative null-check when we are suggesting to inline type checks

Fixes #21097
Fixes #24286

* fix a case where persistent storage registration fails and some clean… (#24458)

* fix a case where persistent storage registration fails and some clean up code around it.

* added readonly

* address PR feedback

* removed comments no longer relevant

* renamed lock name

* moved waiter from diagnostics.dll to features.dll where all interfaces are defined. (#24512)

* put listener change back in (#24120)

* leave old types in legacy folder until partner teams move to new interface

* added legacy waiter to support partner teams

* Remove methods indirecting access to _metadataFileNameToConvertedProjectReference

This field is documented as being written and read from any thread,
but in practice all uses are guarded by an AssertIsForeground(). Thus
we can get rid of the helper methods that are trying to "help" by
locking before accessing the fields, making it really hard to track all
the real uses of it.

* Make method static that doesn't need state

* Fix up tests of P2P to metadata reference conversion

It turns out we had some tests, but the tests were disabled. This was
because the tests weren't working properly anyways: they were calling
into UpdateProjectBinPath which only updated some (but not all) of
the project state. That was an internal helper method that shouldn't
be used by tests. Updating the tests to use
SetBinOutputPathAndRelatedData works better.

* Delete debug-only reference validation

This was some legacy code that tried to verify that the references
we have from the project system match up to what DTE and other sources
say. This was debug-only, and the actual asserts were commented out.
This is deadweight at this point, so delete it.

* added and cleaned up logs around build and live diagnostics. (#24551)

also added RoslynActivityLogger that can be enabled through project-system-tool

* Avoid closure allocations on the BindSyntaxTreeToId fast path

* CS1628 error text mentions in parameters; fixes #24584

* Update optimization data to 2.7.0-beta3-62526-01...

* Small cleanup of completion logic.

* Locate implementations for reference assemblies using the process binding path

* Use GlobalAssemblyCache helper to locate assemblies directly in the GAC

* Update InteractiveEditorFeatures to account for a second definition of GlobalAssemblyCache

* Move to xunit.console for CoreClr tests

Previously we were using xunit.console for desktop tests and dotnet-xunit for our
CoreClr tests. This change unifies us on top of xunit.console (now that it has a
netcoreapp2.0 version available).

* Move unix builds to xunit.runner.console as well

* Fixes 559223

Fix and re-enable test that would catch this error

* Update LanguageServices training data again...

* Get actual directory name, not file

* Fix dir name issue

* Cleanup based on code review feedback

* Check fully-qualified names for SuppressIldasmAttribute and ReferenceAssemblyAttribute
* Use correct reference location, or fail decompilation if it's not available

* Fix typo...

* Don't use inferred member name if that creates duplicates (#24632)

* Fixes #23983

* Added test for unique IDEDiagnosticIDs

* Fixed capitalization on local variable

* Fix `is` and pattern-matching behavior in presence of implicit UD conversion (#24547)

* Fix `is` and pattern-matching behavior in presence of implicit UD conversion
and also an explicit reference conversion. User-defined conversions should
never be considered for `is` and pattern-matching.
Fixes #24522
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants