Skip to content

branch-3.1: [feat](s3client) Use CustomAwsCredentialsProviderChain for aws sdk #56065 #56943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Oct 15, 2025
Merged

branch-3.1: [feat](s3client) Use CustomAwsCredentialsProviderChain for aws sdk #56065 #56943

morrySnow merged 1 commit into from
Oct 15, 2025

Conversation

@swjtu-zhanglei
Copy link
Contributor

@swjtu-zhanglei swjtu-zhanglei commented Oct 14, 2025 •
edited by morrySnow
Loading

picked from #56065

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@swjtu-zhanglei
Copy link
Contributor Author

swjtu-zhanglei commented Oct 14, 2025

pick #56065

@swjtu-zhanglei
Copy link
Contributor Author

swjtu-zhanglei commented Oct 14, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

Cloud UT Coverage Report

Increment line coverage 33.33% (11/33) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 82.49% (1258/1525)
Line Coverage 66.23% (22516/33996)
Region Coverage 67.55% (11249/16654)
Branch Coverage 57.22% (5945/10390)

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

TPC-H: Total hot run time: 32911 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fc88fdb4d5ce21fe85fd98514c4b1903a7dcb740, data reload: false

------ Round 1 ----------------------------------
q1	17896	5554	5509	5509
q2	2027	379	284	284
q3	12374	1253	742	742
q4	10281	876	451	451
q5	8934	2444	2171	2171
q6	181	165	133	133
q7	922	726	619	619
q8	9340	1488	1196	1196
q9	5289	5025	4981	4981
q10	6789	2263	1827	1827
q11	472	282	265	265
q12	350	363	211	211
q13	17793	3631	3029	3029
q14	224	226	215	215
q15	523	472	460	460
q16	425	436	379	379
q17	606	864	359	359
q18	7044	6482	6382	6382
q19	1208	964	539	539
q20	324	344	203	203
q21	2916	2240	1981	1981
q22	1060	1007	975	975
Total cold run time: 106978 ms
Total hot run time: 32911 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5516	5489	5525	5489
q2	231	334	230	230
q3	2247	2686	2318	2318
q4	1323	1773	1332	1332
q5	4488	5025	5041	5025
q6	165	159	126	126
q7	2089	1977	1879	1879
q8	2723	2874	2793	2793
q9	7379	7254	7266	7254
q10	3052	3310	2647	2647
q11	571	527	501	501
q12	706	788	574	574
q13	3406	3826	3195	3195
q14	289	303	280	280
q15	516	479	468	468
q16	459	482	440	440
q17	1260	1749	1267	1267
q18	7590	7467	7220	7220
q19	797	1185	1133	1133
q20	2025	2072	1893	1893
q21	5427	5042	4753	4753
q22	1100	1042	1012	1012
Total cold run time: 53359 ms
Total hot run time: 51829 ms

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

TPC-DS: Total hot run time: 192175 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fc88fdb4d5ce21fe85fd98514c4b1903a7dcb740, data reload: false

query1	957	420	431	420
query2	6246	1939	1866	1866
query3	8680	197	194	194
query4	34061	23754	23581	23581
query5	3604	605	468	468
query6	297	198	181	181
query7	4223	496	335	335
query8	339	256	243	243
query9	9417	2632	2602	2602
query10	446	318	256	256
query11	17961	15424	15363	15363
query12	157	108	103	103
query13	1550	528	407	407
query14	10102	6474	6540	6474
query15	201	197	176	176
query16	7857	617	517	517
query17	1547	757	586	586
query18	2098	449	339	339
query19	234	195	173	173
query20	129	124	130	124
query21	207	130	110	110
query22	4633	4590	4546	4546
query23	35159	34487	33989	33989
query24	7467	2707	2708	2707
query25	515	463	394	394
query26	824	286	178	178
query27	2632	464	356	356
query28	5564	2268	2201	2201
query29	693	612	442	442
query30	235	192	174	174
query31	1053	912	848	848
query32	81	62	60	60
query33	494	386	317	317
query34	774	866	539	539
query35	804	809	745	745
query36	1038	1070	973	973
query37	105	98	70	70
query38	4098	4029	4024	4024
query39	1505	1520	1491	1491
query40	214	117	111	111
query41	48	50	48	48
query42	125	107	102	102
query43	512	506	486	486
query44	1364	832	834	832
query45	192	183	180	180
query46	899	1059	692	692
query47	1966	2024	1943	1943
query48	426	437	346	346
query49	745	516	422	422
query50	690	704	419	419
query51	7340	7220	7250	7220
query52	113	103	97	97
query53	239	270	189	189
query54	555	554	479	479
query55	81	77	81	77
query56	278	293	260	260
query57	1300	1308	1215	1215
query58	225	215	223	215
query59	2999	3203	2961	2961
query60	286	294	262	262
query61	138	115	116	115
query62	827	759	685	685
query63	229	202	191	191
query64	3739	1092	737	737
query65	3408	3302	3278	3278
query66	883	439	329	329
query67	16546	15785	15615	15615
query68	5156	849	566	566
query69	507	330	271	271
query70	1141	1106	1061	1061
query71	379	297	268	268
query72	5353	3742	3745	3742
query73	647	750	356	356
query74	10601	9393	8991	8991
query75	3264	3172	2696	2696
query76	2987	1180	778	778
query77	457	365	278	278
query78	10393	10589	9516	9516
query79	3660	897	593	593
query80	806	514	460	460
query81	495	251	220	220
query82	945	118	88	88
query83	172	160	151	151
query84	242	100	83	83
query85	773	364	301	301
query86	398	316	284	284
query87	4314	4377	4234	4234
query88	5170	2406	2402	2402
query89	415	331	298	298
query90	1789	190	187	187
query91	136	145	109	109
query92	67	54	51	51
query93	2920	895	550	550
query94	774	377	298	298
query95	350	277	267	267
query96	486	616	280	280
query97	3196	3224	3197	3197
query98	225	205	194	194
query99	1520	1417	1327	1327
Total cold run time: 294092 ms
Total hot run time: 192175 ms

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

ClickBench: Total hot run time: 29.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fc88fdb4d5ce21fe85fd98514c4b1903a7dcb740, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.63	0.11	0.10
query5	0.51	0.51	0.53
query6	1.12	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.59	0.49	0.50
query10	0.55	0.54	0.55
query11	0.16	0.11	0.11
query12	0.14	0.11	0.11
query13	0.61	0.62	0.59
query14	0.78	0.77	0.79
query15	0.84	0.86	0.82
query16	0.40	0.38	0.38
query17	1.03	1.01	1.04
query18	0.23	0.22	0.22
query19	1.89	1.86	1.82
query20	0.02	0.01	0.01
query21	15.38	0.96	0.57
query22	0.75	1.08	0.63
query23	14.89	1.40	0.51
query24	3.53	1.65	1.30
query25	0.13	0.17	0.17
query26	0.30	0.14	0.14
query27	0.05	0.05	0.03
query28	13.96	0.98	0.44
query29	12.58	3.88	3.19
query30	0.25	0.09	0.06
query31	2.83	0.58	0.38
query32	3.22	0.54	0.46
query33	3.00	3.02	3.02
query34	16.61	5.20	4.50
query35	4.56	4.55	4.58
query36	0.65	0.50	0.48
query37	0.08	0.06	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 104.06 s
Total hot run time: 29.1 s

@doris-robot
Copy link

doris-robot commented Oct 14, 2025

BE UT Coverage Report

Increment line coverage 86.49% (32/37) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.78% (12875/28125)
Line Coverage 36.66% (115290/314504)
Region Coverage 34.18% (65762/192393)
Branch Coverage 31.20% (34524/110662)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 14, 2025

BE Regression && UT Coverage Report

Increment line coverage 91.89% (34/37) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.40% (21135/27662)
Line Coverage 69.66% (218591/313803)
Region Coverage 67.58% (130612/193280)
Branch Coverage 61.23% (68093/111214)

@morrySnow morrySnow changed the title branch-3.1: [feat](s3client) Use CustomAwsCredentialsProviderChain for aws sdk (#56065) branch-3.1: [feat](s3client) Use CustomAwsCredentialsProviderChain for aws sdk #56065 Oct 15, 2025
@morrySnow morrySnow merged commit a5d34cf into apache:branch-3.1 Oct 15, 2025
22 of 23 checks passed
@morrySnow morrySnow mentioned this pull request Oct 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@swjtu-zhanglei @hello-stephen @doris-robot @morrySnow