Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

NetworkChange should capture AsyncLocals (but not to Timer)#26073

Merged
stephentoub merged 5 commits intodotnet:masterfrom
benaadams:nac-asynclocal
Feb 1, 2018
Merged

NetworkChange should capture AsyncLocals (but not to Timer)#26073
stephentoub merged 5 commits intodotnet:masterfrom
benaadams:nac-asynclocal

Conversation

@benaadams
Copy link
Member

}
if (s_availabilityTimer == null)
{
// Don't capture the current ExecutionContext and its AsyncLocals onto the timer causing them to live forever
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not? What's special about this case that the execution context shouldn't flow into the callback? I can see the argument being made in cases where only infrastructure code is invoked in the callback, but here user code is being invoked.

(And if it is necessary here, the current structure of this is problematic, as it could end up throwing an exception if flow is already suppressed.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its capturing the asynclocals for only the first added delegate onto the timer; not the second, third etc; so it would be the wrong locals.

However I assume if the EC was going to be captured its the addition of to the delegate that should capture them (with each addition being its own context)

s_availabilityChangedSubscribers += value;

Not the single creation of a globally static timer?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the suppression

@wfurt
Copy link
Member

wfurt commented Dec 27, 2017

it looks ok to me. What is practical behavior change with the new/old code?

@benaadams
Copy link
Member Author

The observable behaviour change is if an AsyncLocal with a change event handler was attached it will no longer receive events every time the timer fires https://github.com/dotnet/corefx/issues/25477#issuecomment-346866897 (section "And everything else after here makes no sense" though this is from SqlClient's pool timers)

The main issue it addresses is the timer will capture the AsyncLocal values for the first registration for the entire time there is >= 1 events registered (even if the first registration is removed); extending their lifetime and meaning they cannot be GC'd or finalized. It will also restore the first registration's AsyncLocal's onto every subsequent registrations' callbacks.

I don't think adding to a delegate captures the specific ExecutionContext for the registration; so if they were depended on (and only ever one registration, when it would work) it could be a problem? (as @stephentoub mentions in #26073 (comment))

@benaadams
Copy link
Member Author

benaadams commented Dec 27, 2017

Looking deeper; it looks like the Windows implementation captures the ExecutionContexts individually; then runs the callbacks on them; however the OSX and Linux implementations will only capture the first registration's.

Will need modifications to match Windows behaviour

@wfurt
Copy link
Member

wfurt commented Dec 28, 2017

thanks for the explanation @benaadams

@benaadams benaadams changed the title NetworkChange shouldn't capture AsyncLocals into its Timer [WIP] NetworkChange shouldn't capture AsyncLocals into its Timer Dec 28, 2017
@benaadams
Copy link
Member Author

Marked as WIP as Linux and OSX need to capture the EC per registration (as per Windows)

@davidsh davidsh removed their assignment Jan 2, 2018
@karelz
Copy link
Member

karelz commented Jan 18, 2018

@benaadams what is the next step here?

@benaadams benaadams changed the title [WIP] NetworkChange shouldn't capture AsyncLocals into its Timer [WIP] NetworkChange should capture AsyncLocals into its Timer Jan 18, 2018
@benaadams benaadams changed the title [WIP] NetworkChange should capture AsyncLocals into its Timer [WIP] NetworkChange should capture AsyncLocals (but not to Timer) Jan 18, 2018
@benaadams
Copy link
Member Author

The change is correct; however the event registrations on Linux and OSX should capture the Execution context for each registration and restore the context when firing the event (as per the Windows implementation)

@karelz
Copy link
Member

karelz commented Jan 27, 2018

@benaadams do you have plans to finish it off? If you don't have time now, let's close it, until you have :)
Thanks!

@benaadams benaadams changed the title [WIP] NetworkChange should capture AsyncLocals (but not to Timer) NetworkChange should capture AsyncLocals (but not to Timer) Jan 28, 2018
@benaadams
Copy link
Member Author

@karelz could do with some x-plat deduping; but should be good now

@benaadams
Copy link
Member Author

NETFX x86 failures #26615

private static readonly NetworkAvailabilityEventArgs s_notAvailableEventArgs = new NetworkAvailabilityEventArgs(isAvailable: false);
private static readonly ContextCallback s_runHandlerAvailable = new ContextCallback(RunAvailabilityHandlerAvailable);
private static readonly ContextCallback s_runHandlerNotAvailable = new ContextCallback(RunAvailabilityHandlerNotAvailable);
private static readonly ContextCallback s_runAddressChangedHandler = new ContextCallback(RunAddressChangedHandler);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is adding a fair amount of duplication between the Linux and OSX implementations, and it's also missing this optimization for the Windows implementation. Can you move this shared code into a shared NetworkAddressChange.cs partial file, and then also use these s_{not}AvailableEventArgs from the Windows implementation? More can be deduped later, but we should at least avoid adding more duplication.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deduped; also copied dictionaries under lock for Linux/OSX and skipped trying to add null listeners as per Windows

}

s_addressChangedSubscribers += value;
s_addressChangedSubscribers.TryAdd(value, ExecutionContext.Capture());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. This represents a breaking change from the previous implementation, in that previously if you did:

NetworkAddressChangedEventHandler handler = ...;
NetworkAddress.NetworkAddressChanged += handler;
NetworkAddress.NetworkAddressChanged += handler;

the handler would then be invoked twice, whereas now it'll only be invoked once. But that "only once" behavior is the same behavior that exists on Windows (both core and netfx), so it makes sense to make this match, and I'm not particularly concerned about this causing any issues, not only because it was already this way on Windows, but because you need to be prepared for any number of these notifications to happen due to the nature of the event.

case Interop.Sys.NetworkChangeKind.AddressAdded:
case Interop.Sys.NetworkChangeKind.AddressRemoved:
s_addressChangedSubscribers?.Invoke(null, EventArgs.Empty);
foreach (var subscriber in s_addressChangedSubscribers)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: var => KeyValuePair<NetworkAddressChangedEventHandler, ExecutionContext>

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if (changed)
{
s_availabilityChangedSubscribers?.Invoke(null, new NetworkAvailabilityEventArgs(NetworkInterface.GetIsNetworkAvailable()));
var isAvailable = NetworkInterface.GetIsNetworkAvailable();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var => bool

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


if (ec == null) // Flow supressed
{
handler(null, isAvailable ? s_availableEventArgs : s_notAvailableEventArgs);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move these isAvailable ? a : b checks to be before the loop, rather than doing it for each invocation?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

{
s_availabilityChangedSubscribers?.Invoke(null, new NetworkAvailabilityEventArgs(NetworkInterface.GetIsNetworkAvailable()));
var isAvailable = NetworkInterface.GetIsNetworkAvailable();
foreach (var subscriber in s_availabilityChangedSubscribers)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same var nit

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@benaadams
Copy link
Member Author

Windows x64 failure (Windows.10.Amd64.Open-Debug-x64)

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException
Unhandled Exception of Type Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException
Message :
Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException :
 An internal consistency check failed
Stack Trace :
   at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Cng\src\System\Security\Cryptography\CngKey.Create.cs:line 58
   at Internal.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm) in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Cng\src\Internal\Cryptography\CngAlgorithmCore.cs:line 60
   at System.Security.Cryptography.DSACng.get_Key() in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Cng\src\System\Security\Cryptography\DSACng.Key.cs:line 25
   at System.Security.Cryptography.DSACng.GetDuplicatedKeyHandle() in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Cng\src\System\Security\Cryptography\DSACng.Key.cs:line 52
   at System.Security.Cryptography.DSACng.ComputeQLength() in D:\j\workspace\windows-TGrou---74aa877a\src\Common\src\System\Security\Cryptography\DSACng.SignVerify.cs:line 139
   at System.Security.Cryptography.DSACng.AdjustHashSizeIfNecessaryWithArrayPool(ReadOnlySpan`1& hash) in D:\j\workspace\windows-TGrou---74aa877a\src\Common\src\System\Security\Cryptography\DSACng.SignVerify.cs:line 116
   at System.Security.Cryptography.DSACng.CreateSignature(Byte[] rgbHash) in D:\j\workspace\windows-TGrou---74aa877a\src\Common\src\System\Security\Cryptography\DSACng.SignVerify.cs:line 27
   at System.Security.Cryptography.DSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm) in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Algorithms\src\System\Security\Cryptography\DSA.cs:line 88
   at System.Security.Cryptography.DSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm) in D:\j\workspace\windows-TGrou---74aa877a\src\System.Security.Cryptography.Algorithms\src\System\Security\Cryptography\DSA.cs:line 77
   at System.Security.Cryptography.Dsa.Tests.DSASignVerify_Array.SignData(DSA dsa, Byte[] data, HashAlgorithmName hashAlgorithm) in D:\j\workspace\windows-TGrou---74aa877a\src\Common\tests\System\Security\Cryptography\AlgorithmImplementations\DSA\DSASignVerify.cs:line 14
   at System.Security.Cryptography.Dsa.Tests.DSASignVerify.InvalidKeySize_DoesNotInvalidateKey() in D:\j\workspace\windows-TGrou---74aa877a\src\Common\tests\System\Security\Cryptography\AlgorithmImplementations\DSA\DSASignVerify.cs:line 101

@benaadams
Copy link
Member Author

NETFX x86 Release Build failure

14:53:14       System.IO.Compression.GzipStreamUnitTests.Dispose_WithUnfinishedWriteAsync [FAIL]
14:53:14         Assert.False() Failure
14:53:14         Expected: False
14:53:14         Actual:   True
14:53:14         Stack Trace:
14:53:14              at System.IO.Compression.CompressionStreamUnitTestBase.Dispose_WithUnfinishedWriteAsync()

@benaadams
Copy link
Member Author

benaadams commented Jan 31, 2018

test Windows x64 Debug Build

@benaadams
Copy link
Member Author

test NETFX x86 Release Build

if (value != null)
{
s_addressChangedSubscribers.TryAdd(value, ExecutionContext.Capture());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about doing the value != null check once, and before taking the lock?

add
{
    if (value != null)
    {
        lock (s_gate)
        {
            if (s_socket == 0)
            {
                CreateSocket();
            }

            s_addressChangedSubscribers.TryAdd(value, ExecutionContext.Capture());
        }
    }
}

restoreFlow = true;
}

s_availabilityTimer = new Timer(s_availabilityTimerFiredCallback, null, -1, -1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: for clarity, can we use Timeout.Infinite instead of -1?

remove
{
lock (s_gate)
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do any of this if value is null? Wondering if there should be parity with add.

}
if (s_availabilityTimer == null)

if (s_availabilityTimer == null && value != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some null-related comments as in the other event.

{
changed = s_availabilityHasChanged;
s_availabilityHasChanged = false;
if (s_availabilityChangedSubscribers.Count > 0)
Copy link
Member

@stephentoub stephentoub Jan 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be:

if (changed && s_availabilityChangedSubscribers.Count > 0)

Otherwise we're potentially allocating/copying the dictionary and then not using the copy at all.

And, actually, we don't need changed now. We can just make the condition:

if (s_availabilityHasChanged && s_availabilityChangedSubscribers.Count > 0)

and then below the check just becomes:

if (availabilityChangedSubscribers != null)

}

s_addressChangedSubscribers += value;
if (value != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comments as earlier about null. I'll stop commenting on these.

Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this.

@benaadams
Copy link
Member Author

benaadams commented Jan 31, 2018

Done feedback

Lets tempt the wrath of an outerloop...

@dotnet-bot test Outerloop Windows x64 Debug Build please
@dotnet-bot test Outerloop Linux x64 Debug Build please
@dotnet-bot test Outerloop OSX x64 Debug Build please

@benaadams
Copy link
Member Author

benaadams commented Jan 31, 2018

All sorts of strange errors... Windows x64 Debug Build

Unhandled Exception of Type System.Net.Internals.SocketExceptionFactory+ExtendedSocketException
Message :
System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Device not configured
Stack Trace :
   at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6) in /Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 83
   at System.Net.Dns.GetHostEntry(String hostNameOrAddress) in /Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/src/System.Net.NameResolution/src/System/Net/DNS.cs:line 475
   at System.Net.NameResolution.Tests.GetHostEntryTest.<>c__DisplayClass1_0.<Dns_GetHostEntry_HostString_Ok>b__0() in /Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/src/System.Net.NameResolution/tests/FunctionalTests/GetHostEntryTest.cs:line 26
   at System.Net.NameResolution.Tests.GetHostEntryTest.TestGetHostEntryAsync(Func`1 getHostEntryFunc) in /Users/dotnet-bot/j/workspace/dotnet_corefx/master/osx-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_false_prtest/src/System.Net.NameResolution/tests/FunctionalTests/GetHostEntryTest.cs:line 38
--- End of stack trace from previous location where exception was thrown ---

@benaadams
Copy link
Member Author

@dotnet-bot test this please

@benaadams
Copy link
Member Author

System.IO.Compression.Brotli.Performance.Tests issues across the board it looks like

Output: export XUNIT_PERFORMANCE_MAX_ITERATION_INNER_SPECIFIED=1
Output: chmod +x /home/helixbot/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Payload/dotnet
Output: /home/helixbot/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Payload/dotnet xunit.console.netcore.exe System.IO.Compression.Brotli.Performance.Tests.dll  -xml testResults.xml -parallel none -notrait category=nonnetcoreapptests -notrait category=nonlinuxtests  -notrait category=failing
Output: python DumplingHelper.py collect_dump $\ `pwd`  System.IO.Compression.Brotli.Performance.Tests /mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_true_prtest/bin/runtime/netcoreapp-Linux-Debug-x64/,/mnt/j/workspace/dotnet_corefx/master/linux-TGroup_netcoreapp+CGroup_Debug+AGroup_x64+TestOuter_true_prtest/bin/tests/System.IO.Compression.Brotli.Performance.Tests/netcoreapp-Linux-Debug-x64/,/home/helixbot/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Payload,/lib/x86_64-linux-gnu/libgcc_s.so.1,/lib/x86_64-linux-gnu/libpthread.so.0,/lib/x86_64-linux-gnu/librt.so.1,/usr/lib/x86_64-linux-gnu/libunwind.so.8,/lib/x86_64-linux-gnu/libdl.so.2,/lib/x86_64-linux-gnu/libuuid.so.1,/usr/lib/x86_64-linux-gnu/libunwind-x86_64.so.8,/usr/lib/x86_64-linux-gnu/libstdc++.so.6,/lib/x86_64-linux-gnu/libm.so.6,/lib/x86_64-linux-gnu/libc.so.6,/lib64/ld-linux-x86-64.so.2,/lib/x86_64-linux-gnu/liblzma.so.5
Output: ~/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Work/e3b0b446-0134-4c9f-93c0-acb75d705cc6/Unzip ~/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Work/e3b0b446-0134-4c9f-93c0-acb75d705cc6/Unzip
Output:   File "/home/helixbot/dotnetbuild/work/8e86f5c1-c899-47b3-a72c-0850ca203910/Work/e3b0b446-0134-4c9f-93c0-acb75d705cc6/Unzip/dumpling.py", line 1
Output:     <!DOCTYPE html>
Output:     ^
Output: SyntaxError: invalid syntax
Output: /usr/bin/python: can't open file '/home/helixbot/.dumpling/dumpling.py': [Errno 2] No such file or directory
Output: xUnit.net console test runner (64-bit .NET Core)
Output: Copyright (C) 2014 Outercurve Foundation.
Output: 
Output: Discovering: System.IO.Compression.Brotli.Performance.Tests
Output: Discovered:  System.IO.Compression.Brotli.Performance.Tests
Output: Starting:    System.IO.Compression.Brotli.Performance.Tests

@stephentoub
Copy link
Member

@dotnet-bot test this please

@stephentoub stephentoub merged commit 6000379 into dotnet:master Feb 1, 2018
@karelz karelz added this to the 2.1.0 milestone Feb 4, 2018
@benaadams benaadams deleted the nac-asynclocal branch January 11, 2019 21:35
picenka21 pushed a commit to picenka21/runtime that referenced this pull request Feb 18, 2022
…orefx#26073)

* NetworkChange shouldn't capture AsyncLocals into its Timer

* Capture EC on Linux

* Capture EC on OSX

* Feedback/dedupe platforms

* Better null checking


Commit migrated from dotnet/corefx@6000379
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants