Skip to content

Seamlessly handling Elasticache (AWS) failovers? #1874

@jchowdown

Description

@jchowdown

Hi, is anyone else connecting to a Multi-AZ Elasticache cluster, and is able to seamlessly handle a failover in a way that makes the client block until the connection is reestablished, and then resumes? I'm using these options:

                var options = ConfigurationOptions.Parse(address);
                options.ReconnectRetryPolicy = new ExponentialRetry(250);
                options.SyncTimeout = 10000;
                options.AsyncTimeout = 10000;
                options.AbortOnConnectFail = false;

but when I initiate a failover, I immediately get this exception:

redis_connection_failed_message: Interactive#1@#####.usw2.cache.amazonaws.com:6379 (Idle)
redis_connection_failed_message: Subscription#2@#####.usw2.cache.amazonaws.com:6379 (Idle)

Unhandled Exception:
StackExchange.Redis.RedisConnectionException: No connection is active/available to service this operation: GET pubsub.test.value; SocketClosed (ReadEndOfStream, last-recv: 0) on #####.usw2.cache.amazonaws.com:6379/Subscription, Idle/MarkProcessed, last: SUBSCRIBE, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 21s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 8 of 10 available, in: 0, in-pipe: 0, out-pipe: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.2.50.36290, mc: 1/1/0, mgr: 10 of 10 available, clientName: lax-jchow03-mac, IOCP: (Busy=0,Free=200,Min=16,Max=200), WORKER: (Busy=0,Free=1600,Min=16,Max=1600), v: 2.2.50.36290 ---> StackExchange.Redis.RedisConnectionException: SocketClosed (ReadEndOfStream, last-recv: 0) on #####.usw2.cache.amazonaws.com:6379/Subscription, Idle/MarkProcessed, last: SUBSCRIBE, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 21s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 8 of 10 available, in: 0, in-pipe: 0, out-pipe: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.2.50.36290
--- End of inner exception stack trace ---

Is there another group of settings I'm missing? (I don't think 'connectRetry' is relevant)

Thanks in advance,
Jeff

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions