session,server: refactor the execute statement processing#17555
Merged
tiancaiamao merged 13 commits intopingcap:masterfrom Jun 5, 2020
Merged
session,server: refactor the execute statement processing#17555tiancaiamao merged 13 commits intopingcap:masterfrom
tiancaiamao merged 13 commits intopingcap:masterfrom
Conversation
The executor maybe still refering `txn` inside the Next() function. Do not call txn.Commit() until the executor close.
Codecov Report
@@ Coverage Diff @@
## master #17555 +/- ##
===========================================
Coverage 79.6646% 79.6646%
===========================================
Files 524 524
Lines 142314 142314
===========================================
Hits 113374 113374
Misses 19892 19892
Partials 9048 9048 |
Contributor
Author
|
/run-all-tests PTAL @coocood @jackysp @SunRunAway |
Contributor
Author
|
/run-all-tests |
Contributor
Author
|
/run-all-tests |
Contributor
Author
|
/rebuild |
Contributor
Author
|
/run-all-tests |
Contributor
Author
Member
|
LGTM |
jackysp
reviewed
Jun 5, 2020
| // Execute the physical plan. | ||
| logStmt(stmtNode, s.sessionVars) | ||
| recordSet, err := runStmt(ctx, s, stmt) | ||
| recordSet, err := runStmtWrap(ctx, s, stmt) |
Contributor
There was a problem hiding this comment.
So the binary protocol (execute statement) will still use runStmt? Do you need to fix it?
Contributor
Author
There was a problem hiding this comment.
Yes, it will be handled in the following works.
Contributor
|
/merge |
Contributor
|
/run-all-tests |
Contributor
|
/run-all-tests |
Contributor
|
@tiancaiamao merge failed. |
Contributor
Author
|
/run-all-tests |
2 similar comments
Contributor
Author
|
/run-all-tests |
Contributor
Author
|
/run-all-tests |
Contributor
Author
|
/rebuild |
Contributor
Author
|
/run-mybatis-test |
Member
|
@tiancaiamao could you cherry-pick this PR to release 4.0? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #16817
Problem Summary:
The root cause of the panic in #16817 is the data race.
And the data race exists because we close the transaction immediately in
ExecuteStmt.When using the
RecordSetreturned byExecuteStmt, executor.Next() is called and it maybe still refering the txn, the txn is closed and set to nil however.tidb/session/tidb.go
Line 211 in 0f6b6a2
What is changed and how it works?
We should not commit a txn until the executor stop refering it.
The correct order is:
rsrs.Next(), drain the chunk data and sent result to the mysql clientrs.Close()finishStmtshould be called and txn maybe commit here.What's Changed:
finishStmtout ofrunStmt.execStmtResultbased onRecordSet, override theClosemethod,and call
finishStmtinexecStmtResult.Close()How it Works:
Txn is not commited until the executor close the RecordSet, so there aren't be any concurrent visits of txn between
finishStmtandrecordset.Next, the data race is fixed.Related changes
The refactor in #16056 avoid the multiple return value
[]RecordSet. It makes a solid foundantion towards better code for the session processing.And this commit is a furthur step after that. This commit solve the problem finally:
This commit guarantee the session won't be modify until the chunk data are drained and recordset is closed.
Check List
Tests
runStmtWrapis a copy ofrunStmt, that's because even a tiny modify would meet numerous CI error, I've tried.Keep this commit as simple as possible: just move
finishStmttoRecordSet.CloseAnd after it's merged, I'll clean old
Executemethod (which usesrunStmt).Release note