Skip to content

[fix & opt](orc) ORC-1525: Fix bad read in RleDecoderV2::readByte and Decompress zlib by libdeflate. #53610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Jul 21, 2025
Merged

[fix & opt](orc) ORC-1525: Fix bad read in RleDecoderV2::readByte and Decompress zlib by libdeflate. #53610

dataroaring merged 1 commit into from
Jul 21, 2025
+1 −1

Conversation

@kaka11chen
Copy link
Contributor

@kaka11chen kaka11chen commented Jul 21, 2025 •
edited by morningman
Loading

Cherry-pick #51775

@kaka11chen kaka11chen requested a review from dataroaring as a code owner July 21, 2025 02:37
@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 21, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@kaka11chen
Copy link
Contributor Author

kaka11chen commented Jul 21, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

TPC-H: Total hot run time: 39799 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 04d3c26658afd7a82bc12d06472299e4151f8be9, data reload: false

------ Round 1 ----------------------------------
q1	17747	7174	6592	6592
q2	2082	166	160	160
q3	10688	1119	1192	1119
q4	10442	718	732	718
q5	7711	2902	2866	2866
q6	218	136	128	128
q7	955	625	600	600
q8	9600	1989	1971	1971
q9	8148	6419	6386	6386
q10	7013	2279	2284	2279
q11	450	260	261	260
q12	391	207	211	207
q13	17759	2997	2971	2971
q14	239	206	200	200
q15	525	468	453	453
q16	471	379	371	371
q17	972	544	556	544
q18	7363	6685	6610	6610
q19	1401	1070	1095	1070
q20	487	207	203	203
q21	3879	3097	3130	3097
q22	1135	997	994	994
Total cold run time: 109676 ms
Total hot run time: 39799 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6578	6555	6519	6519
q2	332	232	231	231
q3	2983	2985	2930	2930
q4	2058	1851	1763	1763
q5	5683	5705	5743	5705
q6	208	127	129	127
q7	2178	1804	1809	1804
q8	3381	3555	3502	3502
q9	8838	8938	8853	8853
q10	3564	3551	3602	3551
q11	581	483	494	483
q12	816	575	631	575
q13	7535	3247	3245	3245
q14	307	285	285	285
q15	515	466	487	466
q16	504	463	466	463
q17	1856	1654	1608	1608
q18	8626	7874	8014	7874
q19	1667	1564	1612	1564
q20	2084	1868	1868	1868
q21	5157	5015	4959	4959
q22	1166	1039	999	999
Total cold run time: 66617 ms
Total hot run time: 59374 ms

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

TPC-DS: Total hot run time: 197291 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 04d3c26658afd7a82bc12d06472299e4151f8be9, data reload: false

query1	1298	916	891	891
query2	6291	2006	1957	1957
query3	10970	4330	4622	4330
query4	33104	23470	23449	23449
query5	3511	463	440	440
query6	251	170	174	170
query7	3986	312	316	312
query8	272	225	223	223
query9	9474	2585	2579	2579
query10	451	276	260	260
query11	17725	15240	15285	15240
query12	149	102	102	102
query13	1575	436	423	423
query14	9717	6614	7417	6614
query15	254	183	195	183
query16	8131	566	482	482
query17	1620	593	599	593
query18	2137	333	329	329
query19	333	166	168	166
query20	125	120	109	109
query21	205	105	112	105
query22	4842	4523	4619	4523
query23	34749	34422	34322	34322
query24	11190	2850	2965	2850
query25	685	420	422	420
query26	1232	178	181	178
query27	2318	353	372	353
query28	7242	2153	2219	2153
query29	876	454	454	454
query30	282	167	164	164
query31	1051	824	833	824
query32	95	56	53	53
query33	762	327	280	280
query34	1045	522	523	522
query35	860	738	728	728
query36	1090	928	933	928
query37	134	67	67	67
query38	4179	4001	4055	4001
query39	1560	1479	1548	1479
query40	206	104	102	102
query41	49	48	50	48
query42	114	102	102	102
query43	537	492	475	475
query44	1287	824	825	824
query45	182	169	171	169
query46	1149	723	720	720
query47	2011	1933	1957	1933
query48	476	397	398	397
query49	928	370	385	370
query50	846	423	433	423
query51	7250	7168	7283	7168
query52	100	91	89	89
query53	260	182	187	182
query54	1308	479	486	479
query55	83	78	81	78
query56	273	276	267	267
query57	1315	1187	1199	1187
query58	226	211	207	207
query59	3268	3163	3059	3059
query60	295	265	277	265
query61	120	144	117	117
query62	849	689	700	689
query63	225	195	191	191
query64	4091	675	644	644
query65	3339	3191	3184	3184
query66	1054	291	312	291
query67	15733	15776	15574	15574
query68	4455	583	561	561
query69	424	266	275	266
query70	1134	1115	1115	1115
query71	346	254	254	254
query72	6367	4145	4072	4072
query73	746	343	350	343
query74	10390	9205	9268	9205
query75	3394	2636	2640	2636
query76	2686	1083	1170	1083
query77	386	278	280	278
query78	10569	9638	9546	9546
query79	1730	588	585	585
query80	1128	425	428	425
query81	554	220	224	220
query82	935	86	86	86
query83	227	152	164	152
query84	235	88	80	80
query85	1374	302	293	293
query86	423	294	304	294
query87	4388	4262	4222	4222
query88	3754	2420	2395	2395
query89	416	287	292	287
query90	1843	185	190	185
query91	183	148	149	148
query92	63	49	51	49
query93	2007	558	557	557
query94	824	298	292	292
query95	364	260	261	260
query96	611	286	289	286
query97	3278	3130	3157	3130
query98	211	211	191	191
query99	1522	1305	1312	1305
Total cold run time: 300312 ms
Total hot run time: 197291 ms

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 41.89% (11133/26579)
Line Coverage 32.43% (95372/294095)
Region Coverage 31.56% (49241/156048)
Branch Coverage 27.99% (25238/90184)

@doris-robot
Copy link

doris-robot commented Jul 21, 2025

ClickBench: Total hot run time: 30.14 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 04d3c26658afd7a82bc12d06472299e4151f8be9, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.10	0.10
query5	0.54	0.53	0.53
query6	1.12	0.72	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.52	0.49
query10	0.54	0.55	0.56
query11	0.16	0.11	0.10
query12	0.14	0.10	0.10
query13	0.61	0.60	0.61
query14	0.78	0.79	0.81
query15	0.84	0.82	0.82
query16	0.38	0.40	0.38
query17	1.02	1.07	1.06
query18	0.23	0.22	0.22
query19	1.88	1.77	1.76
query20	0.02	0.01	0.01
query21	15.46	0.59	0.59
query22	2.39	1.79	1.66
query23	16.98	1.13	0.86
query24	3.35	0.94	2.97
query25	0.20	0.15	0.04
query26	0.66	0.14	0.14
query27	0.04	0.04	0.05
query28	8.45	0.52	0.46
query29	12.60	3.24	3.21
query30	0.24	0.06	0.06
query31	2.86	0.39	0.40
query32	3.24	0.46	0.46
query33	2.97	3.00	3.02
query34	17.10	4.50	4.50
query35	4.61	4.60	4.59
query36	0.66	0.48	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.12	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.18 s
Total hot run time: 30.14 s

dataroaring
dataroaring approved these changes Jul 21, 2025
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit cc045b3 into apache:branch-3.0 Jul 21, 2025
22 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dataroaring dataroaring dataroaring approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kaka11chen @hello-stephen @doris-robot @dataroaring