Skip to content

[fix](filecache) fix BytesScannedFromCache/Local always zero #55929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 2 commits into from
Sep 14, 2025
Merged

[fix](filecache) fix BytesScannedFromCache/Local always zero #55929

dataroaring merged 2 commits into from
Sep 14, 2025

Conversation

@freemandealer
Copy link
Contributor

@freemandealer freemandealer commented Sep 11, 2025

forget update profile counters in update_realtime_counters. bug introduced in #52232

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason <- lack of way to validate profile->

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@freemandealer
Copy link
Contributor Author

freemandealer commented Sep 11, 2025

run buildall

@freemandealer
[fix](filecache) fix BytesScannedFromCache/Local always zero
8bc22bb 
forget update profile counters in update_realtime_counters
bug introduced in apache#52232

Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
@freemandealer
Copy link
Contributor Author

freemandealer commented Sep 11, 2025

run buildall

dataroaring
dataroaring approved these changes Sep 11, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Sep 11, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Sep 11, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 11, 2025

PR approved by anyone and no changes requested.

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-H: Total hot run time: 34809 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8bc22bbc4dd5a06354469dc3a7b7c909e280d32f, data reload: false

------ Round 1 ----------------------------------
q1	17623	5207	5086	5086
q2	2021	339	208	208
q3	10216	1263	741	741
q4	10252	1050	520	520
q5	7521	2451	2357	2357
q6	184	172	139	139
q7	909	749	657	657
q8	9358	1336	1102	1102
q9	6943	5080	5070	5070
q10	6953	2378	1995	1995
q11	488	306	276	276
q12	352	364	232	232
q13	17788	3649	3057	3057
q14	237	239	214	214
q15	570	494	490	490
q16	1018	1011	934	934
q17	598	859	361	361
q18	7594	7088	7161	7088
q19	1582	949	578	578
q20	348	350	244	244
q21	3756	3276	2465	2465
q22	1089	1035	995	995
Total cold run time: 107400 ms
Total hot run time: 34809 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5248	5119	5105	5105
q2	256	333	229	229
q3	2200	2635	2332	2332
q4	1337	1784	1362	1362
q5	4219	4570	4613	4570
q6	217	183	146	146
q7	2117	2125	1900	1900
q8	2855	2608	2717	2608
q9	7607	7289	7379	7289
q10	3106	3310	2836	2836
q11	555	539	512	512
q12	689	785	628	628
q13	3547	3993	3289	3289
q14	286	304	282	282
q15	526	497	477	477
q16	1074	1129	1051	1051
q17	1177	1576	1444	1444
q18	7981	7611	7277	7277
q19	819	798	914	798
q20	1901	1998	1806	1806
q21	4730	4399	4297	4297
q22	1079	1072	1018	1018
Total cold run time: 53526 ms
Total hot run time: 51256 ms

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

TPC-DS: Total hot run time: 190149 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8bc22bbc4dd5a06354469dc3a7b7c909e280d32f, data reload: false

query1	1071	444	422	422
query2	6587	1712	1700	1700
query3	6774	225	225	225
query4	26069	23776	23292	23292
query5	4466	709	558	558
query6	352	262	268	262
query7	4665	544	309	309
query8	316	273	254	254
query9	8639	2934	2890	2890
query10	481	346	313	313
query11	15941	15113	14826	14826
query12	169	122	115	115
query13	1675	564	445	445
query14	11114	9364	9451	9364
query15	218	201	196	196
query16	7469	708	468	468
query17	1224	772	634	634
query18	2022	439	347	347
query19	213	202	195	195
query20	144	130	125	125
query21	215	158	114	114
query22	4019	4079	4101	4079
query23	33970	32925	32950	32925
query24	8283	2446	2500	2446
query25	606	527	453	453
query26	1241	293	169	169
query27	2731	540	366	366
query28	4448	2246	2219	2219
query29	796	673	501	501
query30	292	231	200	200
query31	959	820	744	744
query32	95	90	91	90
query33	587	415	361	361
query34	820	924	537	537
query35	831	844	767	767
query36	1020	1072	892	892
query37	135	125	104	104
query38	3579	3572	3471	3471
query39	1529	1416	1441	1416
query40	229	141	135	135
query41	70	69	62	62
query42	140	123	126	123
query43	503	492	465	465
query44	1421	895	891	891
query45	191	178	172	172
query46	926	1070	673	673
query47	1743	1811	1734	1734
query48	409	442	326	326
query49	779	514	429	429
query50	687	738	417	417
query51	3970	4019	3842	3842
query52	126	118	115	115
query53	248	296	206	206
query54	629	622	544	544
query55	96	92	94	92
query56	360	388	340	340
query57	1186	1201	1128	1128
query58	304	305	288	288
query59	2590	2604	2565	2565
query60	365	374	363	363
query61	169	164	165	164
query62	837	738	664	664
query63	242	208	206	206
query64	4517	1181	854	854
query65	4078	3974	3984	3974
query66	1141	499	389	389
query67	15261	15366	15017	15017
query68	7971	968	596	596
query69	516	351	311	311
query70	1462	1311	1296	1296
query71	548	360	348	348
query72	5838	5011	5006	5006
query73	665	628	367	367
query74	9263	9097	9045	9045
query75	3404	3291	2911	2911
query76	3235	1250	790	790
query77	496	425	342	342
query78	9700	9773	8881	8881
query79	2079	891	601	601
query80	715	603	531	531
query81	521	332	234	234
query82	227	175	146	146
query83	272	275	256	256
query84	256	114	97	97
query85	861	480	450	450
query86	388	337	294	294
query87	3747	3800	3663	3663
query88	2938	2249	2250	2249
query89	420	348	306	306
query90	1932	238	232	232
query91	176	169	145	145
query92	96	80	76	76
query93	2252	1005	648	648
query94	704	445	325	325
query95	427	346	340	340
query96	473	663	289	289
query97	2899	3024	2876	2876
query98	259	226	221	221
query99	1350	1451	1305	1305
Total cold run time: 275072 ms
Total hot run time: 190149 ms

@doris-robot
Copy link

doris-robot commented Sep 11, 2025

ClickBench: Total hot run time: 30 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8bc22bbc4dd5a06354469dc3a7b7c909e280d32f, data reload: false

query1	0.06	0.05	0.06
query2	0.10	0.05	0.06
query3	0.26	0.09	0.09
query4	1.61	0.12	0.12
query5	0.30	0.26	0.26
query6	1.23	0.64	0.66
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.60	0.54	0.53
query10	0.59	0.57	0.57
query11	0.17	0.12	0.12
query12	0.15	0.12	0.12
query13	0.63	0.64	0.63
query14	1.03	1.06	1.02
query15	0.89	0.86	0.88
query16	0.44	0.41	0.40
query17	1.05	1.09	1.07
query18	0.22	0.21	0.21
query19	2.00	1.88	1.89
query20	0.02	0.01	0.02
query21	15.40	0.96	0.59
query22	0.81	1.15	0.63
query23	14.97	1.38	0.64
query24	6.72	1.16	0.75
query25	0.46	0.32	0.08
query26	0.59	0.18	0.14
query27	0.07	0.06	0.06
query28	9.54	0.98	0.44
query29	12.58	3.91	3.28
query30	0.32	0.16	0.14
query31	2.84	0.60	0.38
query32	3.25	0.56	0.48
query33	3.20	3.15	3.07
query34	16.11	5.47	4.82
query35	4.95	4.90	4.89
query36	0.70	0.54	0.50
query37	0.11	0.08	0.08
query38	0.07	0.05	0.04
query39	0.04	0.04	0.03
query40	0.17	0.15	0.15
query41	0.09	0.04	0.04
query42	0.05	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 104.52 s
Total hot run time: 30 s

@freemandealer
Copy link
Contributor Author

freemandealer commented Sep 12, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 12, 2025

TPC-H: Total hot run time: 34623 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b7d7747ce7544caf9ce2d4df064c58ae5442c4f, data reload: false

------ Round 1 ----------------------------------
q1	17609	5161	5063	5063
q2	1964	368	209	209
q3	10236	1276	719	719
q4	10236	1027	511	511
q5	7540	2376	2347	2347
q6	186	167	139	139
q7	930	761	637	637
q8	9337	1317	1122	1122
q9	6992	5131	5175	5131
q10	6958	2417	1975	1975
q11	481	310	278	278
q12	363	366	236	236
q13	17793	3666	3044	3044
q14	272	244	216	216
q15	567	491	487	487
q16	1006	1004	963	963
q17	609	866	358	358
q18	7483	7212	7092	7092
q19	1317	960	568	568
q20	342	341	242	242
q21	3675	3136	2307	2307
q22	1044	1038	979	979
Total cold run time: 106940 ms
Total hot run time: 34623 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5225	5085	5066	5066
q2	245	329	234	234
q3	2141	2679	2296	2296
q4	1355	1781	1352	1352
q5	4216	4390	4570	4390
q6	216	183	128	128
q7	2054	2037	1782	1782
q8	2724	2634	2622	2622
q9	7428	7372	7269	7269
q10	3178	3327	2905	2905
q11	561	518	621	518
q12	738	791	601	601
q13	3484	3916	3312	3312
q14	337	303	303	303
q15	513	481	477	477
q16	1114	1117	1042	1042
q17	1167	1558	1431	1431
q18	8092	7631	7806	7631
q19	830	844	855	844
q20	2012	2064	1869	1869
q21	4767	4389	4318	4318
q22	1105	1052	990	990
Total cold run time: 53502 ms
Total hot run time: 51380 ms

@doris-robot
Copy link

doris-robot commented Sep 12, 2025

TPC-DS: Total hot run time: 189266 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b7d7747ce7544caf9ce2d4df064c58ae5442c4f, data reload: false

query1	1064	445	417	417
query2	6618	1711	1677	1677
query3	6755	229	225	225
query4	26500	23648	23209	23209
query5	4395	661	507	507
query6	333	262	245	245
query7	4659	529	307	307
query8	286	260	245	245
query9	8689	2884	2881	2881
query10	493	347	306	306
query11	15889	15128	14739	14739
query12	175	121	122	121
query13	1675	557	436	436
query14	10670	9194	9214	9194
query15	209	199	175	175
query16	7375	695	472	472
query17	1224	810	622	622
query18	2023	443	337	337
query19	218	202	178	178
query20	136	128	120	120
query21	216	134	115	115
query22	4120	4316	4014	4014
query23	33848	33028	32922	32922
query24	8245	2369	2411	2369
query25	608	520	438	438
query26	1229	297	167	167
query27	2723	536	363	363
query28	4426	2246	2224	2224
query29	807	656	492	492
query30	298	235	204	204
query31	942	814	753	753
query32	89	85	81	81
query33	603	394	348	348
query34	821	905	524	524
query35	834	841	773	773
query36	1024	1047	906	906
query37	126	118	100	100
query38	3553	3491	3483	3483
query39	1507	1424	1442	1424
query40	232	142	133	133
query41	65	63	61	61
query42	134	117	119	117
query43	507	499	481	481
query44	1405	875	854	854
query45	186	177	174	174
query46	888	1073	647	647
query47	1767	1801	1745	1745
query48	392	435	328	328
query49	782	531	414	414
query50	675	740	403	403
query51	4003	3834	3901	3834
query52	119	118	114	114
query53	249	295	203	203
query54	633	637	572	572
query55	97	94	97	94
query56	360	355	357	355
query57	1218	1199	1139	1139
query58	329	298	291	291
query59	2572	2701	2535	2535
query60	393	356	363	356
query61	162	160	155	155
query62	839	715	651	651
query63	238	199	200	199
query64	4503	1151	822	822
query65	4032	3983	3986	3983
query66	1171	470	364	364
query67	15406	15688	15233	15233
query68	7278	975	585	585
query69	509	334	303	303
query70	1416	1324	1270	1270
query71	555	357	320	320
query72	5932	5023	5109	5023
query73	693	685	366	366
query74	9189	9102	9024	9024
query75	3389	3303	2782	2782
query76	3230	1232	840	840
query77	520	417	350	350
query78	9648	9796	8946	8946
query79	2277	860	594	594
query80	665	600	531	531
query81	517	275	231	231
query82	236	172	150	150
query83	273	260	250	250
query84	271	114	103	103
query85	905	458	437	437
query86	387	313	268	268
query87	3795	3735	3597	3597
query88	3698	2230	2228	2228
query89	426	322	285	285
query90	1989	245	238	238
query91	160	173	136	136
query92	97	80	75	75
query93	2005	1017	643	643
query94	709	444	332	332
query95	410	343	335	335
query96	494	631	284	284
query97	2930	2993	2859	2859
query98	271	234	213	213
query99	1364	1467	1298	1298
Total cold run time: 275000 ms
Total hot run time: 189266 ms

@doris-robot
Copy link

doris-robot commented Sep 12, 2025

ClickBench: Total hot run time: 30.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b7d7747ce7544caf9ce2d4df064c58ae5442c4f, data reload: false

query1	0.05	0.05	0.04
query2	0.09	0.05	0.05
query3	0.25	0.09	0.08
query4	1.61	0.12	0.12
query5	0.27	0.27	0.25
query6	1.18	0.67	0.65
query7	0.03	0.02	0.02
query8	0.05	0.04	0.05
query9	0.61	0.52	0.52
query10	0.58	0.57	0.59
query11	0.17	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.63
query14	1.03	1.04	1.04
query15	0.88	0.85	0.87
query16	0.40	0.42	0.39
query17	1.05	1.07	1.04
query18	0.22	0.20	0.20
query19	1.92	1.80	1.87
query20	0.02	0.01	0.02
query21	15.39	0.94	0.58
query22	0.77	1.39	0.96
query23	14.79	1.35	0.61
query24	6.79	2.37	0.79
query25	0.57	0.11	0.19
query26	0.63	0.16	0.15
query27	0.05	0.06	0.05
query28	9.46	0.90	0.44
query29	12.58	3.93	3.23
query30	0.31	0.13	0.12
query31	2.83	0.60	0.41
query32	3.25	0.55	0.49
query33	3.31	3.15	3.17
query34	15.94	5.47	4.86
query35	4.94	4.90	4.87
query36	0.71	0.52	0.50
query37	0.10	0.07	0.08
query38	0.07	0.05	0.04
query39	0.04	0.02	0.02
query40	0.19	0.16	0.15
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 104.08 s
Total hot run time: 30.21 s

@doris-robot
Copy link

doris-robot commented Sep 12, 2025

BE UT Coverage Report

Increment line coverage 33.33% (1/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.21% (17424/33373)
Line Coverage 37.43% (158362/423104)
Region Coverage 32.01% (120793/377399)
Branch Coverage 33.38% (52996/158765)

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 12, 2025

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 70.85% (23121/32632)
Line Coverage 57.23% (241543/422090)
Region Coverage 52.55% (200951/382390)
Branch Coverage 54.22% (86461/159466)

@dataroaring dataroaring merged commit 66577bb into apache:master Sep 14, 2025
26 of 28 checks passed
freemandealer added a commit to freemandealer/doris that referenced this pull request Oct 30, 2025
@freemandealer
[fix](filecache) fix BytesScannedFromCache/Local always zero (apache#…
1402f28 
…55929)

forget update profile counters in update_realtime_counters. bug
introduced in apache#52232
freemandealer added a commit to freemandealer/doris that referenced this pull request Oct 30, 2025
@freemandealer
[fix](filecache) fix BytesScannedFromCache/Local always zero (apache#…
0704eb6 
…55929)

forget update profile counters in update_realtime_counters. bug
introduced in apache#52232
@morrySnow morrySnow mentioned this pull request Nov 3, 2025
Merged
morrySnow pushed a commit that referenced this pull request Nov 3, 2025
@freemandealer
branch-3.1: [fix](filecache) fix BytesScannedFromCache/Local always z…
cf9a884 
…ero #55929 (#57536)

picked from #55929
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou gavinchou approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.3-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@freemandealer @hello-stephen @doris-robot @gavinchou @dataroaring @morrySnow