Skip to content

[Log](topn) Log query id in topn two read and change the timeout to uery timeout #53450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
HappenLee merged 1 commit into from
Jul 18, 2025
Merged

[Log](topn) Log query id in topn two read and change the timeout to uery timeout #53450

HappenLee merged 1 commit into from
Jul 18, 2025

Conversation

@HappenLee
Copy link
Contributor

@HappenLee HappenLee commented Jul 17, 2025

What problem does this PR solve?

  1. Log query id in topn two read
  2. change the timeout to use query timeout

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@HappenLee
Copy link
Contributor Author

HappenLee commented Jul 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-H: Total hot run time: 34386 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5a60fccd480c4323a2cfe19299bdfc1d2e117308, data reload: false

------ Round 1 ----------------------------------
q1	17660	5377	5121	5121
q2	1930	286	175	175
q3	10404	1314	740	740
q4	10240	1021	565	565
q5	7697	2417	2362	2362
q6	185	161	130	130
q7	920	745	595	595
q8	9300	1326	1136	1136
q9	7061	5167	5201	5167
q10	6924	2401	1976	1976
q11	504	313	285	285
q12	349	368	215	215
q13	17814	3759	3102	3102
q14	229	232	219	219
q15	548	488	487	487
q16	424	447	386	386
q17	596	885	404	404
q18	7596	7250	7124	7124
q19	1284	957	604	604
q20	352	349	237	237
q21	4148	3298	2407	2407
q22	1027	1019	949	949
Total cold run time: 107192 ms
Total hot run time: 34386 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5281	5178	5163	5163
q2	250	323	223	223
q3	2165	2724	2338	2338
q4	1408	1813	1359	1359
q5	4493	4561	4498	4498
q6	212	172	134	134
q7	2095	1978	1820	1820
q8	2617	2537	2538	2537
q9	7354	7359	7326	7326
q10	3100	3345	2972	2972
q11	594	511	516	511
q12	711	813	680	680
q13	3531	4041	3421	3421
q14	297	298	278	278
q15	528	470	482	470
q16	460	495	470	470
q17	1173	1621	1635	1621
q18	8075	7696	7679	7679
q19	877	847	920	847
q20	1976	1949	1808	1808
q21	4826	4389	4288	4288
q22	1070	1018	1012	1012
Total cold run time: 53093 ms
Total hot run time: 51455 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

TPC-DS: Total hot run time: 187390 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5a60fccd480c4323a2cfe19299bdfc1d2e117308, data reload: false

query1	1010	372	398	372
query2	6544	1682	1687	1682
query3	6743	215	216	215
query4	26631	23636	23352	23352
query5	4398	593	453	453
query6	309	227	211	211
query7	4632	492	284	284
query8	264	218	217	217
query9	8629	2620	2584	2584
query10	481	320	277	277
query11	15751	14989	14832	14832
query12	154	113	104	104
query13	1631	522	409	409
query14	9089	5845	5727	5727
query15	209	205	170	170
query16	7198	613	484	484
query17	1189	703	586	586
query18	1965	413	307	307
query19	205	190	167	167
query20	132	126	118	118
query21	218	124	106	106
query22	4089	4147	4086	4086
query23	34324	33176	33145	33145
query24	8488	2387	2430	2387
query25	567	490	429	429
query26	1102	267	153	153
query27	2753	512	340	340
query28	4318	2156	2135	2135
query29	787	596	465	465
query30	295	222	192	192
query31	943	858	775	775
query32	79	136	64	64
query33	555	377	306	306
query34	808	834	506	506
query35	806	811	738	738
query36	963	1004	920	920
query37	110	98	72	72
query38	4170	4191	4024	4024
query39	1480	1429	1427	1427
query40	238	118	104	104
query41	56	52	51	51
query42	113	101	99	99
query43	494	506	480	480
query44	1328	826	824	824
query45	177	167	163	163
query46	836	1003	619	619
query47	1752	1836	1745	1745
query48	391	424	319	319
query49	738	482	379	379
query50	630	686	402	402
query51	5440	5686	5440	5440
query52	113	102	91	91
query53	227	255	178	178
query54	563	562	511	511
query55	80	82	84	82
query56	317	314	293	293
query57	1215	1208	1128	1128
query58	264	256	268	256
query59	2613	2672	2529	2529
query60	341	333	329	329
query61	132	129	129	129
query62	803	726	649	649
query63	218	193	180	180
query64	4324	1016	674	674
query65	4299	4142	4197	4142
query66	1139	435	374	374
query67	15824	15554	15853	15554
query68	8399	901	534	534
query69	473	307	275	275
query70	1265	1173	1137	1137
query71	467	314	302	302
query72	5686	4818	4836	4818
query73	713	611	353	353
query74	8968	9226	8994	8994
query75	3855	3207	2720	2720
query76	3631	1159	728	728
query77	793	372	301	301
query78	10246	10141	9320	9320
query79	2781	789	566	566
query80	621	533	526	526
query81	468	252	224	224
query82	453	126	97	97
query83	289	260	243	243
query84	289	114	84	84
query85	802	432	318	318
query86	333	311	297	297
query87	4444	4550	4421	4421
query88	3237	2284	2277	2277
query89	407	316	286	286
query90	1933	208	205	205
query91	143	146	111	111
query92	77	63	61	61
query93	2069	935	594	594
query94	674	407	318	318
query95	374	307	286	286
query96	485	577	274	274
query97	2844	2724	2685	2685
query98	231	205	205	205
query99	1450	1420	1285	1285
Total cold run time: 277584 ms
Total hot run time: 187390 ms

@doris-robot
Copy link

doris-robot commented Jul 17, 2025

ClickBench: Total hot run time: 33.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5a60fccd480c4323a2cfe19299bdfc1d2e117308, data reload: false

query1	0.05	0.03	0.04
query2	0.08	0.04	0.04
query3	0.25	0.08	0.07
query4	1.62	0.11	0.10
query5	0.45	0.44	0.41
query6	1.17	0.66	0.68
query7	0.03	0.02	0.02
query8	0.06	0.03	0.04
query9	0.61	0.53	0.51
query10	0.60	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.61
query14	0.82	0.84	0.83
query15	0.91	0.88	0.89
query16	0.39	0.39	0.41
query17	1.07	1.07	1.06
query18	0.24	0.21	0.21
query19	1.94	1.90	1.86
query20	0.02	0.01	0.02
query21	15.36	0.94	0.56
query22	0.75	1.23	0.76
query23	14.76	1.39	0.63
query24	6.96	1.37	1.32
query25	0.53	0.27	0.08
query26	0.52	0.17	0.13
query27	0.06	0.05	0.06
query28	10.35	0.90	0.44
query29	12.57	4.00	3.33
query30	3.09	3.05	3.00
query31	2.83	0.59	0.40
query32	3.25	0.58	0.48
query33	3.06	3.10	3.20
query34	16.11	5.42	4.83
query35	4.81	4.80	4.89
query36	0.68	0.51	0.49
query37	0.10	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.17	0.14	0.15
query41	0.09	0.02	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.04
Total cold run time: 107.45 s
Total hot run time: 33.26 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 57.41% (15768/27466)
Line Coverage 46.18% (141892/307245)
Region Coverage 35.67% (106777/299344)
Branch Coverage 38.18% (47140/123453)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 17, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 81.20% (21881/26946)
Line Coverage 73.80% (226393/306775)
Region Coverage 61.51% (188882/307075)
Branch Coverage 65.24% (81450/124843)

@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Jul 18, 2025

PR approved by anyone and no changes requested.

@HappenLee HappenLee merged commit 7735ad6 into apache:master Jul 18, 2025
26 of 29 checks passed
@HappenLee HappenLee mentioned this pull request Jul 22, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jul 23, 2025
@HappenLee
[Log](topn) Log query id in topn two read and change the timeout to u…
e66150e 
…ery timeout (#52717)

cherry pick #53450
@HappenLee HappenLee mentioned this pull request Jul 24, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Gabriel39 Gabriel39 Gabriel39 approved these changes

@zhangstar333 zhangstar333 zhangstar333 approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@HappenLee @hello-stephen @doris-robot @Gabriel39 @zhangstar333 @dataroaring @morrySnow