Skip to content

[Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and restart #57104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Oct 17, 2025
Merged

[Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and restart #57104

dataroaring merged 1 commit into from
Oct 17, 2025

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Oct 17, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

  1. create a table with auto-increment column
  2. FE do checkpoint and generate image file
  3. FE restart and load meta from the image file
  4. AutoIncrementGenerator.nextId are set to batchEndId(which is -1) in AutoIncrementGenerator.gsonPostProcess()

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Oct 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1 bobhan1 marked this pull request as ready for review October 17, 2025 09:11
@bobhan1
Copy link
Contributor Author

bobhan1 commented Oct 17, 2025

run buildall

dataroaring
dataroaring approved these changes Oct 17, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring added usercase Important user case type label dev/3.1.x dev/4.0.x labels Oct 17, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 17, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 17, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

TPC-DS: Total hot run time: 190286 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b21b504417df1421736110fdae67df3e23380364, data reload: false

query1	1067	425	407	407
query2	6582	1720	1687	1687
query3	6758	227	224	224
query4	26819	23588	23537	23537
query5	5018	637	477	477
query6	342	262	248	248
query7	4653	511	297	297
query8	335	273	276	273
query9	8725	2580	2595	2580
query10	557	350	289	289
query11	15611	15033	14916	14916
query12	193	121	113	113
query13	1692	571	452	452
query14	11569	9198	9311	9198
query15	263	191	174	174
query16	7847	683	480	480
query17	1718	818	649	649
query18	2338	463	363	363
query19	683	224	216	216
query20	141	143	137	137
query21	235	137	125	125
query22	4890	5052	4736	4736
query23	35341	33804	33796	33796
query24	8881	2554	2494	2494
query25	603	548	480	480
query26	1336	284	178	178
query27	2681	528	376	376
query28	4232	2223	2173	2173
query29	812	658	511	511
query30	312	263	210	210
query31	933	868	812	812
query32	102	75	83	75
query33	630	388	340	340
query34	892	908	537	537
query35	810	867	811	811
query36	1082	1051	873	873
query37	127	114	84	84
query38	3613	3498	3428	3428
query39	1458	1437	1583	1437
query40	217	135	122	122
query41	66	59	57	57
query42	125	111	114	111
query43	476	493	479	479
query44	1353	827	822	822
query45	194	186	169	169
query46	854	998	625	625
query47	1776	1783	1712	1712
query48	416	429	316	316
query49	772	493	435	435
query50	660	691	405	405
query51	3914	3988	3913	3913
query52	111	111	98	98
query53	248	270	195	195
query54	602	589	529	529
query55	93	87	85	85
query56	340	341	297	297
query57	1192	1201	1126	1126
query58	295	273	279	273
query59	2573	2710	2448	2448
query60	353	348	324	324
query61	156	156	160	156
query62	790	759	652	652
query63	238	202	189	189
query64	4498	1151	865	865
query65	4091	3975	3942	3942
query66	1091	427	341	341
query67	15418	14965	15046	14965
query68	9549	893	595	595
query69	485	325	319	319
query70	1369	1309	1260	1260
query71	510	359	314	314
query72	5507	4807	4830	4807
query73	681	579	349	349
query74	9253	9056	8853	8853
query75	4437	3392	2863	2863
query76	4300	1167	753	753
query77	1038	402	312	312
query78	9679	9828	8850	8850
query79	3058	838	583	583
query80	714	563	508	508
query81	485	256	230	230
query82	268	161	137	137
query83	297	273	246	246
query84	298	113	88	88
query85	862	462	431	431
query86	370	315	320	315
query87	3833	3732	3692	3692
query88	2864	2221	2236	2221
query89	439	320	298	298
query90	2136	230	226	226
query91	158	159	131	131
query92	86	71	68	68
query93	2280	983	641	641
query94	681	439	351	351
query95	409	331	323	323
query96	492	588	279	279
query97	2932	3003	2885	2885
query98	234	219	214	214
query99	1483	1413	1300	1300
Total cold run time: 285980 ms
Total hot run time: 190286 ms

@doris-robot
Copy link

doris-robot commented Oct 17, 2025

ClickBench: Total hot run time: 30.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b21b504417df1421736110fdae67df3e23380364, data reload: false

query1	0.06	0.05	0.04
query2	0.09	0.05	0.06
query3	0.25	0.08	0.08
query4	1.61	0.11	0.12
query5	0.28	0.25	0.24
query6	1.15	0.65	0.64
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.63	0.52	0.51
query10	0.59	0.60	0.59
query11	0.17	0.11	0.11
query12	0.15	0.12	0.11
query13	0.62	0.63	0.61
query14	1.02	1.02	1.04
query15	0.87	0.84	0.85
query16	0.40	0.41	0.39
query17	1.07	1.07	1.04
query18	0.21	0.20	0.19
query19	1.96	1.90	1.85
query20	0.02	0.02	0.02
query21	15.42	0.90	0.58
query22	0.76	1.37	0.74
query23	14.81	1.39	0.63
query24	7.13	1.55	1.14
query25	0.47	0.18	0.10
query26	0.64	0.16	0.13
query27	0.08	0.07	0.06
query28	9.34	1.35	0.93
query29	12.55	3.95	3.28
query30	0.28	0.14	0.12
query31	2.82	0.61	0.39
query32	3.24	0.56	0.47
query33	3.10	3.17	3.05
query34	16.20	5.47	4.81
query35	4.89	4.91	4.88
query36	0.70	0.50	0.50
query37	0.10	0.08	0.07
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.17	0.15
query41	0.09	0.03	0.03
query42	0.03	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 104.19 s
Total hot run time: 30.7 s

Hastyshell
Hastyshell approved these changes Oct 17, 2025
View reviewed changes
Copy link
Collaborator

@Hastyshell Hastyshell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 17, 2025

FE Regression Coverage Report

Increment line coverage 100.00% (1/1) 🎉
Increment coverage report
Complete coverage report

@dataroaring dataroaring merged commit b276af3 into apache:master Oct 17, 2025
36 of 37 checks passed
github-actions bot pushed a commit that referenced this pull request Oct 17, 2025
@bobhan1
[Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and re…
541df2e 
…start (#57104)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
1. create a table with auto-increment column
2. FE do checkpoint and generate image file
3. FE restart and load meta from the image file
4. `AutoIncrementGenerator.nextId` are set to `batchEndId`(which is -1)
in `AutoIncrementGenerator.gsonPostProcess()`


### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Oct 17, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Oct 17, 2025
@bobhan1
[Fix](autoinc) Fix wrong auto inc value after FE do checkpoint and re…
29d1d8d 
…start (#57104)

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
1. create a table with auto-increment column
2. FE do checkpoint and generate image file
3. FE restart and load meta from the image file
4. `AutoIncrementGenerator.nextId` are set to `batchEndId`(which is -1)
in `AutoIncrementGenerator.gsonPostProcess()`


### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Oct 17, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 18, 2025
@github-actions @bobhan1
branch-4.0: [Fix](autoinc) Fix wrong auto inc value after FE do check…
a6a2878 
…point and restart #57104 (#57119)

Cherry-picked from #57104

Co-authored-by: bobhan1 <baohan@selectdb.com>
morrySnow pushed a commit that referenced this pull request Oct 22, 2025
@github-actions @bobhan1
branch-3.1: [Fix](autoinc) Fix wrong auto inc value after FE do check…
57cde86 
…point and restart #57104 (#57118)

Cherry-picked from #57104

Co-authored-by: bobhan1 <baohan@selectdb.com>
Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Nov 12, 2025
@selectdb-robot @bobhan1
branch-selectdb-doris-3.0: [Fix](autoinc) Fix wrong auto inc value af…
76db926 
…ter FE do checkpoint and restart apache#57104 (apache#5635)

cherry-picks from apache#57104
Related to apache#5188

Co-authored-by: bobhan1 <baohan@selectdb.com>
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request Dec 26, 2025
@bobhan1
(selectdb-cloud-4.0): [Fix](autoinc) Fix wrong auto inc value after F…
9c3929b 
…E do checkpoint and restart (apache#57104) (apache#6289)

pick apache#57104
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

+2 more reviewers

@Hastyshell Hastyshell Hastyshell approved these changes

@0AyanamiRei 0AyanamiRei 0AyanamiRei approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.2-merged dev/4.0.1-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@bobhan1 @Thearas @doris-robot @hello-stephen @Hastyshell @dataroaring @0AyanamiRei @yiguolei @morrySnow