-
Notifications
You must be signed in to change notification settings - Fork 444
Multi Subnet Failover #362
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
96c0732 to
3f06dc9
Compare
|
Any news on the status of this? I tried it with a service I'm developing to connect to my load-balanced mssql database and it continues to occasionally time out. I might try chrislukk's branch |
|
@mitant Do you have any more information on when these timeouts happen? |
|
Whats the status of this PR? The inability to set |
|
@takotuesday Would you mind taking this code and going on a test drive with it? @chrislukkk hasn't reported back. |
|
Guys, the problem is not that you can't set "multisubnetfailover=true", it's that the functionality doesn't appear to exist in this driver package to behave in the way that multisubnetfailover=true is supposed to indicate. |
|
@arthurschreiber The behavior that I observed in my testing was that it would behave the same as if the functionality wasn't implemented--that is if I pointed the connection at the "load balancing" host it would occasionally time out. If I point at the db instance behind the load balancing host, it will work flawlessly, but then it's not load balanced anymore. Issue #396 has a similar issue but for some reason they can't connect at all. I personally was able to connect and perform operations for some amount of time, which later would timeout. |
|
I'm on a project that we had to connect to a cluster DB. I'm using Sequelize as the connector wrapper, and it wasn't working until I took this code and manually put it in the node_modules/tedious folder. It's been in there for about a month, and we're just about out of UAT with no connection issues so far. So as far as my experience goes, this is working good. It solved the issue I was having immediately. The project will go to Production in a few weeks so it'll get stress-tested then if that's what y'all are looking for. |
|
@jblaketc Thank you so much for reporting back and letting me know that this is working for you! @mitant Did you use a checkout of this Pull Request in your testing? As long as this PR is not merged, |
|
I'm getting some good results. Earlier in the day pre-pull I was getting db timeouts on startup of my loopback project (which depends on tedious). I haven't observed any timeouts recently. As I do more dev work today I'll make a note if I run into any more issues p.s. to anyone watching this thread, I used the following to load this pr |
|
I have tested this and so far looks promising. After some initial issues with connection, i can reliably run my app tests and pass. p.s. Thanks for the load link @mitant |
use => syntax for passing callbacks so that 'this' is preserved only use multiConnect when the address count is > 1
2. fix if else format 3. concise addresses check
This extracts all the logic related to how the low-level network connection to a TDS server is established into a separate `Connector` class.
3f06dc9 to
cdbdae1
Compare
|
Thanks everyone who tested this! ❤️ It looks like this is stable enough to be merged into |
|
@arthurschreiber I tried running this on my local machine against Node 0.10 and am able to repro. I see the error below. In binary-insert-test.js, test.done() is being called from 'end' callback. Is it possible that this PR is somehow causing the callback to not be invoked? I tried adding callbacks for 'close' and 'error' but that didn't fix it. I'll look at this more later but sharing in case there is something here that may be obvious to you. For some reason this part of the output is not there in on the failure link. binary-insert-test:
To fix this, make sure all tests call test.done() |
|
@tvrprasad Thanks for checking this out! I've not been able to reproduce this so far, but the output you pasted at least gives me some sort of direction! 😄 I'll try to take a deeper look into this after the weekend - hopefully we can find out what's wrong here so this can be merged and released soon! |
|
@arthurschreiber Just for FYI, I was able to repro with Node version v0.10.46, running on Windows version 10.0.10586. |
|
@takotuesday I just pushed another change that would need review from @tvrprasad, then we should be good to go and release this as 1.15.0. @tvrprasad Would you mind taking another look? I added |
src/instance-lookup.js
Outdated
| socket.close(); | ||
|
|
||
| if (port) { | ||
| return callback(undefined, port); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're firing socket.send for all the IP addresses returned at line 86. You could end up invoking this multiple times, right?
src/instance-lookup.js
Outdated
| socket.on('message', onMessage); | ||
| socket.send(request, 0, request.length, SQL_SERVER_BROWSER_PORT, server); | ||
|
|
||
| if (multiSubnetFailover) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MultiSubnetFailover is handled differently here vs in Connector. In connector, we call lookupAll for both cases and then fire requests in parallel or in sequence depending on whether multiSubnetFailover succeeds or not. Whereas here, we only try one IP address for multiSubnetFailover == false. I think we should keep these consistent.
Also we're doing lookupAll twice for instance-lookup code path. Once in instanceLookup() method and then again in connectOnPort(). We should only need to do this once I think. One way to do that would to invoke lookupAll in connect() method for all cases and then pass addresses as a parameter for connectOnPort() method as well as instance-lookup().
One other suggestion for the instance-lookup case would be to use the same IP address that returned the port, in connectOnPort. I'm not sure if it's mandated but seems safe to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One other suggestion for the instance-lookup case would be to use the same IP address that returned the port, in connectOnPort. I'm not sure if it's mandated but seems safe to do.
Turns out, this is actually the wrong thing to do. What you're doing is correct, it's valid for the IP address that responded with the port to not be hosting a SQL Server instance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also we're doing lookupAll twice for instance-lookup code path. Once in instanceLookup() method and then again in connectOnPort(). We should only need to do this once I think. One way to do that would to invoke lookupAll in connect() method for all cases and then pass addresses as a parameter for connectOnPort() method as well as instance-lookup().
I'd expect that the server browser response contains IP-addresses, not DNS host names, so the connection would be done without another DNS lookup.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That sounds reasonable.
|
@arthurschreiber I think we should support UDPv6 to stay consistent with the non-instance name based code path. Also, I added a couple of other review comments. Thanks for the updates! |
comprehensive unit testing. - Refactored UDP code to send message to SQL server browser into a separate class 'Sender' class. Somewhat similar to Connector class. - Added comprehensive unit tests, stubbing with Sinon. - Wrapped code in instance-lookup.js in class to make the code more unit testable. - Currently unit tests for instance-lookup are pretty light. Added more comprehensive tests. - This is a squash of several commits from the branch: - https://github.com/tvrprasad/tedious/tree/multisubnet-failover-udpv6
|
Sent a PR to address remaining issues with MSF. #502 |
Folded it into ParallelSendStrategy.
Extensive refactoring of instance-lookup code to support UDP v6 and
…tisubnet-failover # Conflicts: # src/connection.js
|
@arthurschreiber The sender tests are failing in 0.10 and 0.12. I'll take a look. |
Replaced with EventEmitter.listeners.
EventEmitter.listenerCount is not supported in 0.10 and 0.12.
…tisubnet-failover # Conflicts: # src/connection.js
|
This is awwwwwwwwwwweessoommmmmeee, thank you @arthurschreiber! |
|
Releasing this as 1.15.0 now. |
|
Also, thank you to everyone who helped getting this over the finish line 🏁 ! |
|
This is great! Thanks :-) |
|
Hi @arthurschreiber, |
|
@amihanov, you should be able to just pass the MultiSubnetFailover setting as a dialect option when initializing Sequelize. |
|
@renaudholcombe, Yes!! thanks. it worked great. Here's the code (in case some can use it): var Sequelize = require ('sequelize');
var config = {
adapter: 'mssql',
host: 'myHost',
user: 'myUser',
password: 'myPassword',
port: 1433,
database: 'myDatabase',
dialectOptions: {
multiSubnetFailover: true
},
pool: {
max: 5,
min: 0,
idle: 10000
},
maxConcurrentQueries: 100,
logging: false
};
var sequelize = new Sequelize(config.database, config.user, config.password, {
host: config.host,
port: config.port,
dialect: config.adapter,
dialectOptions: config.dialectOptions,
pool: config.pool,
maxConcurrentQueries: config.maxConcurrentQueries,
logging: config.logging
});
sequelize
.authenticate()
.then(() => {
console.log('Connection has been established successfully.');
})
.catch(err => {
console.error('Unable to connect to the database:', err);
}); |
This is based on the work done by @chrislukkk over in #346.
This PR extracts the whole logic for how a connection is established into a separate
Connectorclass. It also changes the way how connections are established to fully follow the Microsoft specification.multiSubnetFailover: falseare established by looking up all IP Addresses from DNS and trying to connect serially.multiSubnetFailover: trueare established by looking up all IP Addresses from DNS and opening a socket to all of them in parallel. The socket that connects first will be used, all other sockets will be destroyed.@chrislukkk Thanks for your hard work on the initial implementation of this. Please take a look. I'll add extensive unit tests ASAP, but this should be working just fine.