executor: fix load data losing connection when batch_dml_size is set#22724
executor: fix load data losing connection when batch_dml_size is set#22724ti-srebot merged 7 commits intopingcap:masterfrom guo-shaoge:master
Conversation
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
No release note, Please follow https://github.com/pingcap/community/blob/master/contributors/release-note-checker.md |
|
/rebuild |
|
/run-all-tests |
| dbt.mustExec("create table t(c1 bigint auto_random primary key, c2 bigint, c3 bigint)") | ||
| dbt.mustExec(fmt.Sprintf("load data local infile %q into table t (c2, c3)", path)) | ||
| rows := dbt.mustQuery("select count(*) from t") | ||
| cli.checkRows(c, rows, "50000") |
There was a problem hiding this comment.
We also need to check the correctness of the data.
Maybe we can use select bit_xor(c1), bit_xor(c2), bit_xor(c3) from t
There was a problem hiding this comment.
done. (only c2 and c3 is verified. Because c1 is auto_random column, no way to know its value)
server/tidb_test.go
Outdated
| ts.runTestLoadDataForListColumnPartition2(c) | ||
| } | ||
|
|
||
| func (ts *tidbTestSerialSuite) TestLoadDataBatchDML(c *C) { |
There was a problem hiding this comment.
Add a comment for this test, e.g.
// fix issue22540|
/run-all-tests |
|
Have we tested the performance regression? |
|
/merge |
|
/run-all-tests |
|
cherry pick to release-4.0 in PR #22736 |
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
|
cherry pick to release-5.0-rc in PR #22737 |
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
What problem does this PR solve?
Issue Number: Fix #22540
Problem Summary:
When tidb_dml_batch_size is a relatively small(eg: 128/64), load data into a table which has a auto_random column may got lost connection error.
What is changed and how it works?
What's Changed:
How it Works:
load data use two routine:
Commit routine will invalid txn after task is committed. And before generate a new txn, process stream routine may use that invalid txn to generate audo random value. So we add a lock to protect txn, make sure process stream routine use a valid txn.
Related changes
Check List
Tests
Side effects
Release note