This repository was archived by the owner on Aug 23, 2023. It is now read-only.
Peer query speculative fixes and improvements#1430
Merged
Conversation
When using peerQuerySpeculative queries, if a request to a peer fails, instead of failing and aborting all requests, simply retry the request on another peer for the shard.
robert-milan
approved these changes
Aug 14, 2019
shanson7
reviewed
Aug 14, 2019
| } | ||
|
|
||
| if resp.err != nil { | ||
| // check if there is another peer for this shardGroup. If so try it. |
Collaborator
There was a problem hiding this comment.
The reason I originally didn't retry errors, is that many errors aren't effective to retry. Things like bad requests will still fail with the peers. I would be interested to see how often this is actually helpful.
Contributor
Author
There was a problem hiding this comment.
Between when cluster.MembersForSpeculativeQuery() returned the list of peers and when the requests where actually run, nodes could go offline. We have been seeing bursts of failures in clusters that handle a high volume of requests because of this.
Contributor
There was a problem hiding this comment.
no instance should issue a request that is considered a bad request by a peer. if we see that, it is a bug to be fixed.
Dieterbe
added a commit
that referenced
this pull request
Sep 2, 2019
Dieterbe
added a commit
that referenced
this pull request
Sep 17, 2019
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixup context handling for intra-cluster requests. The function for handling requests was using a depreciated approach for cancelling requests.
update peerQuerySpeculative to retry other peers in a shardGroup when a request fails.