Skip to content

branch-3.1: [Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and restart #57104 #57118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Oct 22, 2025
Merged

branch-3.1: [Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and restart #57104 #57118

morrySnow merged 1 commit into from
Oct 22, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Oct 17, 2025

Cherry-picked from #57104

@bobhan1
[Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and re…
541df2e 
…start (#57104)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
1. create a table with auto-increment column
2. FE do checkpoint and generate image file
3. FE restart and load meta from the image file
4. `AutoIncrementGenerator.nextId` are set to `batchEndId`(which is -1)
in `AutoIncrementGenerator.gsonPostProcess()`


### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot requested a review from morrySnow as a code owner October 17, 2025 14:55
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Oct 17, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

TPC-H: Total hot run time: 32547 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 541df2eabfdca86864e7ec8a9c32536c97954cd5, data reload: false

------ Round 1 ----------------------------------
q1	17638	5526	5413	5413
q2	2014	382	271	271
q3	12224	1220	756	756
q4	10227	864	456	456
q5	8145	2377	2111	2111
q6	182	161	134	134
q7	875	731	604	604
q8	9343	1423	1132	1132
q9	5107	4971	4865	4865
q10	6779	2288	1868	1868
q11	469	283	262	262
q12	332	349	215	215
q13	17793	3586	3020	3020
q14	234	222	220	220
q15	530	464	468	464
q16	413	431	379	379
q17	593	850	353	353
q18	6870	6484	6305	6305
q19	1219	945	510	510
q20	336	336	212	212
q21	2966	2211	2002	2002
q22	1047	1021	995	995
Total cold run time: 105336 ms
Total hot run time: 32547 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5549	5483	5485	5483
q2	240	330	234	234
q3	2281	2676	2339	2339
q4	1382	1752	1371	1371
q5	4405	4916	4968	4916
q6	168	164	137	137
q7	2086	1975	1840	1840
q8	2609	2825	2700	2700
q9	7226	7221	7284	7221
q10	3017	3316	2734	2734
q11	588	526	500	500
q12	683	732	646	646
q13	3356	3791	3188	3188
q14	290	301	265	265
q15	524	467	478	467
q16	438	490	433	433
q17	1228	1705	1260	1260
q18	7570	7390	7242	7242
q19	809	1088	1104	1088
q20	2013	2086	1931	1931
q21	5339	4951	4586	4586
q22	1115	1092	1056	1056
Total cold run time: 52916 ms
Total hot run time: 51637 ms

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

TPC-DS: Total hot run time: 192510 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 541df2eabfdca86864e7ec8a9c32536c97954cd5, data reload: false

query1	972	379	380	379
query2	6227	1916	1924	1916
query3	8713	200	195	195
query4	33459	23582	23749	23582
query5	3660	587	453	453
query6	284	188	188	188
query7	4189	479	303	303
query8	300	248	232	232
query9	9242	2607	2575	2575
query10	472	325	266	266
query11	17976	15391	15243	15243
query12	156	111	110	110
query13	1554	545	427	427
query14	8937	7711	6803	6803
query15	261	193	184	184
query16	8027	627	536	536
query17	1588	821	594	594
query18	2173	425	324	324
query19	242	218	177	177
query20	129	123	123	123
query21	213	128	114	114
query22	4668	4566	4402	4402
query23	35216	34427	34049	34049
query24	7269	2689	2728	2689
query25	521	494	436	436
query26	1174	289	179	179
query27	1999	482	370	370
query28	5565	2200	2192	2192
query29	810	615	458	458
query30	244	191	157	157
query31	990	962	822	822
query32	81	58	56	56
query33	509	363	319	319
query34	754	852	514	514
query35	782	825	768	768
query36	1012	1075	975	975
query37	106	90	66	66
query38	4061	4088	3968	3968
query39	1535	1499	1508	1499
query40	208	127	110	110
query41	50	50	46	46
query42	126	104	101	101
query43	490	526	496	496
query44	1297	843	829	829
query45	185	177	179	177
query46	876	1052	661	661
query47	1964	2015	1900	1900
query48	412	444	350	350
query49	798	497	408	408
query50	657	704	423	423
query51	7365	7379	7189	7189
query52	110	98	91	91
query53	226	257	195	195
query54	541	550	461	461
query55	86	79	87	79
query56	274	269	261	261
query57	1280	1275	1229	1229
query58	235	219	214	214
query59	3048	3188	3017	3017
query60	295	305	290	290
query61	120	114	113	113
query62	825	748	714	714
query63	226	203	191	191
query64	4577	989	637	637
query65	3406	3306	3329	3306
query66	1005	407	313	313
query67	16552	15779	15748	15748
query68	7774	815	533	533
query69	481	302	271	271
query70	1165	1123	1106	1106
query71	384	290	266	266
query72	5657	3711	3778	3711
query73	627	739	362	362
query74	10295	9372	9035	9035
query75	3175	3157	2718	2718
query76	3209	1192	762	762
query77	611	357	274	274
query78	10370	10490	9767	9767
query79	3494	908	600	600
query80	690	530	430	430
query81	501	254	218	218
query82	592	122	91	91
query83	156	161	146	146
query84	236	98	88	88
query85	789	382	297	297
query86	382	317	319	317
query87	4390	4359	4230	4230
query88	5119	2427	2440	2427
query89	412	340	305	305
query90	1837	193	193	193
query91	137	141	108	108
query92	70	58	56	56
query93	2211	873	530	530
query94	674	428	303	303
query95	359	283	276	276
query96	490	613	288	288
query97	3187	3290	3129	3129
query98	228	217	197	197
query99	1445	1417	1284	1284
Total cold run time: 294451 ms
Total hot run time: 192510 ms

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

ClickBench: Total hot run time: 28.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 541df2eabfdca86864e7ec8a9c32536c97954cd5, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.03
query3	0.24	0.07	0.06
query4	1.63	0.11	0.11
query5	0.51	0.51	0.50
query6	1.12	0.74	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.57	0.51	0.50
query10	0.55	0.56	0.56
query11	0.15	0.11	0.10
query12	0.14	0.11	0.12
query13	0.63	0.60	0.59
query14	0.78	0.79	0.79
query15	0.84	0.84	0.83
query16	0.37	0.42	0.37
query17	1.07	1.07	1.03
query18	0.24	0.23	0.23
query19	1.88	1.89	1.90
query20	0.02	0.02	0.01
query21	15.36	0.92	0.60
query22	0.74	0.81	0.65
query23	15.10	1.40	0.55
query24	3.41	1.67	0.68
query25	0.22	0.29	0.05
query26	0.31	0.16	0.14
query27	0.05	0.06	0.04
query28	13.40	1.05	0.44
query29	12.62	3.93	3.33
query30	0.25	0.08	0.07
query31	2.84	0.61	0.39
query32	3.23	0.54	0.46
query33	2.99	3.06	3.01
query34	16.56	5.20	4.49
query35	4.59	4.57	4.51
query36	0.63	0.49	0.51
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.14	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.68 s
Total hot run time: 28.73 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 17, 2025

FE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@morrySnow morrySnow merged commit 57cde86 into branch-3.1 Oct 22, 2025
23 checks passed
@github-actions github-actions bot deleted the auto-pick-57104-branch-3.1 branch October 22, 2025 02:22
@morrySnow morrySnow mentioned this pull request Oct 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @morrySnow @dataroaring @bobhan1