Skip to content

[Fix](warmup) Fix wrongly skip download task for warmup job when rowsets have multi segments #56663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Sep 30, 2025
Merged

[Fix](warmup) Fix wrongly skip download task for warmup job when rowsets have multi segments #56663

dataroaring merged 1 commit into from
Sep 30, 2025

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Sep 30, 2025

What problem does this PR solve?

introduced in #54611

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 30, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor

freemandealer commented Sep 30, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 30, 2025

TPC-DS: Total hot run time: 190729 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d19fa6ed76b269e0d313bd7762940edb4975ba15, data reload: false

query1	1070	457	418	418
query2	6590	1699	1707	1699
query3	6766	229	224	224
query4	26673	23820	23084	23084
query5	4953	691	494	494
query6	355	256	235	235
query7	4658	499	293	293
query8	321	267	259	259
query9	8724	2595	2578	2578
query10	521	352	283	283
query11	15543	15138	14802	14802
query12	201	126	116	116
query13	1686	558	445	445
query14	11805	9348	9391	9348
query15	252	190	175	175
query16	7805	667	478	478
query17	1593	766	665	665
query18	2055	463	368	368
query19	296	202	186	186
query20	140	133	127	127
query21	295	139	125	125
query22	4715	4863	4699	4699
query23	35195	33971	33738	33738
query24	8384	2533	2526	2526
query25	564	543	484	484
query26	1557	290	172	172
query27	2715	525	366	366
query28	4389	2190	2187	2187
query29	800	626	491	491
query30	298	250	205	205
query31	1016	872	798	798
query32	84	75	76	75
query33	603	394	366	366
query34	832	881	544	544
query35	853	846	834	834
query36	1008	1038	917	917
query37	121	109	83	83
query38	3526	3499	3524	3499
query39	1502	1414	1421	1414
query40	213	131	112	112
query41	65	57	58	57
query42	117	110	104	104
query43	480	495	478	478
query44	1327	824	819	819
query45	188	182	167	167
query46	844	1012	641	641
query47	1759	1807	1734	1734
query48	393	444	313	313
query49	786	551	422	422
query50	657	705	406	406
query51	3900	3930	3985	3930
query52	107	109	97	97
query53	240	271	195	195
query54	600	590	534	534
query55	91	95	83	83
query56	325	317	307	307
query57	1177	1190	1106	1106
query58	277	276	262	262
query59	2547	2694	2583	2583
query60	343	347	331	331
query61	152	149	148	148
query62	796	730	698	698
query63	233	192	187	187
query64	4113	1135	833	833
query65	4122	3955	4013	3955
query66	995	437	367	367
query67	15431	15481	15128	15128
query68	9091	898	599	599
query69	492	334	288	288
query70	1443	1316	1291	1291
query71	506	369	329	329
query72	6043	4914	4946	4914
query73	704	594	355	355
query74	9122	9060	9036	9036
query75	4382	3419	2813	2813
query76	3630	1182	762	762
query77	958	409	321	321
query78	9819	9996	8984	8984
query79	1560	798	606	606
query80	683	570	523	523
query81	507	267	233	233
query82	460	172	142	142
query83	278	267	248	248
query84	255	116	92	92
query85	877	463	415	415
query86	348	313	307	307
query87	3744	3785	3725	3725
query88	2932	2263	2218	2218
query89	421	332	306	306
query90	2066	222	225	222
query91	157	170	143	143
query92	84	67	64	64
query93	1103	972	644	644
query94	696	443	351	351
query95	407	326	317	317
query96	482	574	287	287
query97	2952	2994	2881	2881
query98	229	216	210	210
query99	1455	1426	1306	1306
Total cold run time: 280628 ms
Total hot run time: 190729 ms

@bobhan1 bobhan1 changed the title [Fix](warmup) Fix wrongly skip download task for warmup job [Fix](warmup) Fix wrongly skip download task for warmup job when rowsets have multi segments Sep 30, 2025
@doris-robot
Copy link

doris-robot commented Sep 30, 2025

ClickBench: Total hot run time: 30.76 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d19fa6ed76b269e0d313bd7762940edb4975ba15, data reload: false

query1	0.05	0.05	0.05
query2	0.08	0.06	0.07
query3	0.25	0.09	0.08
query4	1.61	0.11	0.12
query5	0.28	0.27	0.25
query6	1.18	0.66	0.65
query7	0.03	0.03	0.02
query8	0.05	0.05	0.05
query9	0.64	0.52	0.52
query10	0.58	0.58	0.58
query11	0.19	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.64	0.62
query14	1.02	1.03	1.04
query15	0.85	0.84	0.85
query16	0.39	0.39	0.41
query17	1.06	1.08	1.04
query18	0.21	0.20	0.20
query19	1.96	1.82	1.80
query20	0.01	0.02	0.02
query21	15.43	0.92	0.57
query22	0.76	1.25	0.75
query23	14.74	1.37	0.65
query24	6.97	1.35	1.06
query25	0.46	0.23	0.15
query26	0.66	0.16	0.14
query27	0.06	0.05	0.05
query28	10.18	1.36	0.91
query29	12.63	3.94	3.26
query30	0.27	0.13	0.10
query31	2.83	0.59	0.39
query32	3.24	0.55	0.47
query33	3.10	3.13	3.12
query34	16.05	5.49	4.82
query35	4.91	4.94	4.90
query36	0.70	0.51	0.50
query37	0.10	0.07	0.07
query38	0.06	0.05	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.16
query41	0.08	0.03	0.04
query42	0.04	0.04	0.03
query43	0.05	0.04	0.03
Total cold run time: 104.75 s
Total hot run time: 30.76 s

@bobhan1
Copy link
Contributor Author

bobhan1 commented Sep 30, 2025

run cloud_p0

freemandealer
freemandealer approved these changes Sep 30, 2025
View reviewed changes
Copy link
Contributor

@freemandealer freemandealer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Sep 30, 2025

PR approved by anyone and no changes requested.

dataroaring
dataroaring approved these changes Sep 30, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 30, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 30, 2025

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

doris-robot commented Sep 30, 2025

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.44% (17653/33666)
Line Coverage 37.64% (160339/425998)
Region Coverage 32.13% (122283/380627)
Branch Coverage 33.51% (53631/160029)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 30, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/18) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.07% (18757/32864)
Line Coverage 42.37% (179994/424827)
Region Coverage 36.89% (142023/385034)
Branch Coverage 37.98% (61002/160622)

@bobhan1
Copy link
Contributor Author

bobhan1 commented Sep 30, 2025

run p0

@bobhan1
Copy link
Contributor Author

bobhan1 commented Sep 30, 2025

run cloud_p0

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 30, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/18) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.07% (18757/32864)
Line Coverage 42.37% (179994/424827)
Region Coverage 36.89% (142023/385034)
Branch Coverage 37.98% (61002/160622)

1 similar comment
@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 30, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/18) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.07% (18757/32864)
Line Coverage 42.37% (179994/424827)
Region Coverage 36.89% (142023/385034)
Branch Coverage 37.98% (61002/160622)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 30, 2025

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/18) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.08% (18759/32864)
Line Coverage 42.37% (180010/424827)
Region Coverage 36.83% (141821/385034)
Branch Coverage 37.98% (61007/160622)

@dataroaring dataroaring merged commit 9cb7ba8 into apache:master Sep 30, 2025
31 of 32 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 30, 2025
@bobhan1
[Fix](warmup) Fix wrongly skip download task for warmup job when rows…
7a9f7e4 
…ets have multi segments (#56663)

### What problem does this PR solve?

introduced in #54611

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Sep 30, 2025
Merged
github-actions bot pushed a commit that referenced this pull request Sep 30, 2025
@bobhan1
[Fix](warmup) Fix wrongly skip download task for warmup job when rows…
52039d2 
…ets have multi segments (#56663)

### What problem does this PR solve?

introduced in #54611

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot mentioned this pull request Sep 30, 2025
Merged
yiguolei pushed a commit that referenced this pull request Oct 1, 2025
@github-actions @bobhan1
branch-4.0: [Fix](warmup) Fix wrongly skip download task for warmup j…
cc1914a 
…ob when rowsets have multi segments #56663 (#56681)

Cherry-picked from #56663

Co-authored-by: bobhan1 <baohan@selectdb.com>
dwdwqfwe pushed a commit to dwdwqfwe/doris that referenced this pull request Oct 4, 2025
@bobhan1 @dwdwqfwe
[Fix](warmup) Fix wrongly skip download task for warmup job when rows…
04cecff 
…ets have multi segments (apache#56663)

### What problem does this PR solve?

introduced in apache#54611

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
morrySnow pushed a commit that referenced this pull request Oct 11, 2025
@github-actions @bobhan1
branch-3.1: [Fix](warmup) Fix wrongly skip download task for warmup j…
ec9c05d 
…ob when rowsets have multi segments #56663 (#56680)

Cherry-picked from #56663

Co-authored-by: bobhan1 <baohan@selectdb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

+1 more reviewer

@freemandealer freemandealer freemandealer approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.2-merged dev/4.0.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@bobhan1 @hello-stephen @freemandealer @doris-robot @dataroaring @yiguolei @morrySnow