Functional impact
"Connections Current" performance counters specific to each transport cannot be trusted for diagnosing connection issues.
Steps to reproduce
Here's some code:
Startup.cs
using System;
using Microsoft.AspNet.SignalR;
using Microsoft.AspNet.SignalR.Transports;
using Owin;
namespace SignalRTransportCounterIssue
{
public class Startup
{
static Startup()
{
var transportManager = GlobalHost.DependencyResolver.Resolve<ITransportManager>() as TransportManager;
transportManager.Remove("foreverFrame");
// To see the same thing happen for SSE falling back to long polling, comment out this line:
transportManager.Remove("serverSentEvents");
}
public void Configuration(IAppBuilder app)
{
app.Map("/signalr", map =>
{
map.Use(async (ctx, continuation) =>
{
await continuation();
// Make web sockets fail on the client so that the client falls back to long polling:
if (ctx.Request.Path.Value.StartsWith("/connect") && (ctx.Request.Query["transport"] == "webSockets"))
throw new Exception();
});
map.RunSignalR();
});
}
}
}test.html
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title></title>
<script src="Scripts/jquery-1.6.4.js" type="text/javascript"></script>
<script src="Scripts/jquery.signalR-2.2.0.js" type="text/javascript"></script>
<script type="text/javascript">
var connection = $.hubConnection();
connection.stateChanged(function(e) {
if (e.newState === $.signalR.connectionState.disconnected) {
$("#connectButton").text("Connect");
} else {
$("#connectButton").text("Disconnect");
}
});
$(function() {
$("#connectButton").click(function() {
if ($(this).text() === "Connect") {
connection.start();
} else {
connection.stop();
}
});
});
</script>
</head>
<body>
<div>
<button id="connectButton" style="display: inline-block">Connect</button>
</div>
</body>
</html>
- Make sure SignalR performance counters are installed.
- Create an ASP .NET SignalR app using the code above and build it.
- Create a new web site in IIS on a free port and point it to this app.
- Edit the application pool settings for this web site and set the identity to NetworkService so that it has permission to access the performance counters.
- Open a browser that supports web sockets or server sent events and navigate to test.html.
- Click the "Connect" button. This will spin up SignalR and cause the perf counter instance to be created.
- Close the page and recycle the application pool so we can start with zeros.
- Open perfmon and add the SignalR performance counters for the web site instance to the "Performance Monitor" tool.
- Switch to "Report" graph style.
- Once again, navigate to test.html and click "Connect".
- Click "Disconnect".
Expected result
Since no one is connected anymore, all transport connection counters should say 0.
Actual result
One counter (WebSockets or ServerSentEvents) says 1, and LongPolling says 2,147,483,649.
Further technical details
We are trying to diagnose why our SignalR application thinks there are more connections than there actually are. The application is listening to OnConnected, OnReconnected, and OnDisconnected on the hub, and it keeps track of connection IDs that are connected and which transport they belong to. When we compared our data to the SignalR performance counters, we noticed a rather obvious issue:
The "Connections Current LongPolling" counter is wrong; we don't have over 2 million long pollers. In fact, "Connections Current" reports a total of 2,940 connections, so obviously the math doesn't add up.
What does add up is if you consider 2,147,483,665 to be a negative number, but with something gone terribly wrong in bitwise handling. If you assume that it is a 32-bit number and zero out the sign bit, you get 17. Make that a negative number (yes, I know that is not actually how you would do a bitwise negate operation, but bear with me), and you get -17. Add in the counts from the other transports: -17 + 334 + 2,623 = 2,940. This consistently checks out in all of my testing. Further, a look at the graph for this counter makes it reasonable to think that it is actually a negative number:
We see here that when the process started, it immediately got many long pollers who reconnected, but then the number steadily goes down for some unknown reason until it goes negative.
I've actually been able to reproduce this issue in a new app. It happens when a user first tries to connect using web sockets or server sent events but ends up falling back to long polling. At that time, the counter for the original transport is not decremented; neither is the counter for long polling incremented. Then, once the long poller disconnects, the long polling counter is decremented. See steps to reproduce above.
Functional impact
"Connections Current" performance counters specific to each transport cannot be trusted for diagnosing connection issues.
Steps to reproduce
Here's some code:
Startup.cs
test.html
Expected result
Since no one is connected anymore, all transport connection counters should say 0.
Actual result
One counter (WebSockets or ServerSentEvents) says 1, and LongPolling says 2,147,483,649.
Further technical details
We are trying to diagnose why our SignalR application thinks there are more connections than there actually are. The application is listening to OnConnected, OnReconnected, and OnDisconnected on the hub, and it keeps track of connection IDs that are connected and which transport they belong to. When we compared our data to the SignalR performance counters, we noticed a rather obvious issue:
The "Connections Current LongPolling" counter is wrong; we don't have over 2 million long pollers. In fact, "Connections Current" reports a total of 2,940 connections, so obviously the math doesn't add up.
What does add up is if you consider 2,147,483,665 to be a negative number, but with something gone terribly wrong in bitwise handling. If you assume that it is a 32-bit number and zero out the sign bit, you get 17. Make that a negative number (yes, I know that is not actually how you would do a bitwise negate operation, but bear with me), and you get -17. Add in the counts from the other transports: -17 + 334 + 2,623 = 2,940. This consistently checks out in all of my testing. Further, a look at the graph for this counter makes it reasonable to think that it is actually a negative number:
We see here that when the process started, it immediately got many long pollers who reconnected, but then the number steadily goes down for some unknown reason until it goes negative.
I've actually been able to reproduce this issue in a new app. It happens when a user first tries to connect using web sockets or server sent events but ends up falling back to long polling. At that time, the counter for the original transport is not decremented; neither is the counter for long polling incremented. Then, once the long poller disconnects, the long polling counter is decremented. See steps to reproduce above.