transport: prevent deadlock in transport Close when GoAway write hangs#7662
Merged
purnesh42H merged 3 commits intogrpc:masterfrom Oct 8, 2024
Merged
transport: prevent deadlock in transport Close when GoAway write hangs#7662purnesh42H merged 3 commits intogrpc:masterfrom
purnesh42H merged 3 commits intogrpc:masterfrom
Conversation
…WhenGoAwayWriteHangs
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #7662 +/- ##
==========================================
+ Coverage 81.89% 81.96% +0.06%
==========================================
Files 361 361
Lines 27818 27823 +5
==========================================
+ Hits 22782 22805 +23
+ Misses 3847 3833 -14
+ Partials 1189 1185 -4
|
Contributor
|
Is there an existing issue filed which needs to be linked? |
Contributor
Author
|
@purnesh42H yes, added it in description. |
Contributor
|
Some general comments here:
|
easwars
reviewed
Oct 1, 2024
easwars
approved these changes
Oct 7, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7606.
Couple of recent changes worth noting here:
loopyWriterto exit (after enqueueing the above GOAWAY frame on thecontrolbuf). This was done to ensure that the client transport shutdown can complete in the face of a hanging network connection that blocks forever when attempting to write the above GOAWAY frameDescription of the deadlock:
controlbuf,http2Client.Closecallshttp2Client.GetGoAwayReasonto fetch the last GOAWAY's debug message, and the latter attempts to grabhttp2Client.mu.http2Client.outgoingGoAwayHandlerholdshttp2Client.muwhen it is attempting to write the GOAWAY frame. So, if the underlying network connection is hanging, this method will not release the mutex, and thereforehttp2Client.GetGoAwayReasonwill not be able to grab the same mutex, and therebyhttp2Client.Closewill deadlock.RELEASE NOTES: