Skip to content

branch-3.1: [fix](load) fix execute copy task fail #56551 #56589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morrySnow merged 1 commit into from
Sep 29, 2025
Merged

branch-3.1: [fix](load) fix execute copy task fail #56551 #56589

morrySnow merged 1 commit into from
Sep 29, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Sep 28, 2025

Cherry-picked from #56551

@sollhui
[fix](load) fix execute copy task fail (#56551)
366687c 
### What problem does this PR solve?

introduce by #55498

Fix execute copy task fail:
```
Caused by: org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Cannot invoke "java.util.Optional.isPresent()" because the return value of "org.apache.doris.nereids.trees.plans.co
mmands.info.CopyFromDesc.getFileFilterExpr()" is null
        ... 13 more
Caused by: java.lang.NullPointerException: Cannot invoke "java.util.Optional.isPresent()" because the return value of "org.apache.doris.nereids.trees.plans.commands.info.CopyFromDesc.getFileFilterExpr()" is null
        at org.apache.doris.nereids.trees.plans.commands.info.CopyIntoInfo.doValidate(CopyIntoInfo.java:262) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.info.CopyIntoInfo.validate(CopyIntoInfo.java:195) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.nereids.trees.plans.commands.CopyIntoCommand.run(CopyIntoCommand.java:57) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:668) ~[doris-fe.jar:1.2-SNAPSHOT]
        ... 12 more

```

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@github-actions github-actions bot requested a review from morrySnow as a code owner September 28, 2025 13:46
@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 28, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Sep 28, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Sep 28, 2025

TPC-H: Total hot run time: 32803 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 366687c69af99c4084e016ac28a6d62da5b27fa8, data reload: false

------ Round 1 ----------------------------------
q1	17592	5483	5432	5432
q2	2012	400	274	274
q3	12251	1240	744	744
q4	10329	870	450	450
q5	9751	2402	2111	2111
q6	187	166	134	134
q7	901	736	616	616
q8	9354	1441	1226	1226
q9	5197	4959	4949	4949
q10	6755	2264	1869	1869
q11	466	277	261	261
q12	343	367	218	218
q13	17773	3587	3031	3031
q14	229	222	212	212
q15	520	477	472	472
q16	432	423	381	381
q17	621	869	363	363
q18	6911	6296	6408	6296
q19	1211	974	567	567
q20	325	353	206	206
q21	2941	2175	2012	2012
q22	1025	1019	979	979
Total cold run time: 107126 ms
Total hot run time: 32803 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5606	5498	5520	5498
q2	238	323	230	230
q3	2242	2693	2354	2354
q4	1375	1837	1363	1363
q5	4385	5014	5021	5014
q6	172	162	131	131
q7	2110	1944	1855	1855
q8	2699	2897	2760	2760
q9	7480	7298	7364	7298
q10	3127	3360	2845	2845
q11	572	511	497	497
q12	733	766	619	619
q13	3502	3817	3282	3282
q14	295	325	282	282
q15	515	467	485	467
q16	452	511	448	448
q17	1247	1776	1309	1309
q18	7741	7364	7252	7252
q19	824	1175	1103	1103
q20	2015	2056	1897	1897
q21	5441	4905	4566	4566
q22	1106	1034	1022	1022
Total cold run time: 53877 ms
Total hot run time: 52092 ms

@doris-robot
Copy link

doris-robot commented Sep 28, 2025

TPC-DS: Total hot run time: 192477 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 366687c69af99c4084e016ac28a6d62da5b27fa8, data reload: false

query1	959	401	379	379
query2	6239	1922	1930	1922
query3	8689	201	201	201
query4	33811	24354	23749	23749
query5	3745	605	472	472
query6	287	187	195	187
query7	4188	514	306	306
query8	301	238	233	233
query9	9368	2603	2596	2596
query10	476	319	261	261
query11	18115	15601	15051	15051
query12	152	106	116	106
query13	1550	556	434	434
query14	8859	7748	7268	7268
query15	249	186	178	178
query16	8087	656	494	494
query17	1561	780	612	612
query18	2226	421	327	327
query19	250	194	171	171
query20	128	124	119	119
query21	204	138	110	110
query22	4577	4546	4460	4460
query23	34908	34300	33720	33720
query24	7520	2767	2724	2724
query25	537	492	455	455
query26	1156	287	178	178
query27	2123	476	357	357
query28	5183	2285	2223	2223
query29	772	595	472	472
query30	252	195	167	167
query31	996	901	851	851
query32	91	85	58	58
query33	507	383	333	333
query34	738	856	521	521
query35	782	806	746	746
query36	1020	1066	934	934
query37	102	91	71	71
query38	4021	4057	3939	3939
query39	1574	1470	1515	1470
query40	208	120	110	110
query41	49	50	48	48
query42	122	109	104	104
query43	529	514	500	500
query44	1375	851	842	842
query45	188	181	181	181
query46	900	1072	701	701
query47	1968	1944	1882	1882
query48	403	449	361	361
query49	767	526	419	419
query50	679	695	442	442
query51	7205	7308	7156	7156
query52	101	101	95	95
query53	245	256	188	188
query54	566	546	471	471
query55	83	79	89	79
query56	280	289	281	281
query57	1263	1286	1210	1210
query58	240	227	215	215
query59	3100	3238	3069	3069
query60	310	294	266	266
query61	111	135	114	114
query62	801	773	688	688
query63	238	202	203	202
query64	4471	1017	647	647
query65	3345	3308	3290	3290
query66	1001	415	304	304
query67	16558	15737	15490	15490
query68	7504	813	549	549
query69	496	303	272	272
query70	1158	1119	1090	1090
query71	370	289	283	283
query72	5772	3687	3900	3687
query73	651	749	358	358
query74	10729	9176	9049	9049
query75	3197	3136	2680	2680
query76	3017	1197	778	778
query77	490	358	286	286
query78	10493	10565	9731	9731
query79	3190	892	598	598
query80	596	512	444	444
query81	519	259	226	226
query82	559	121	89	89
query83	165	164	149	149
query84	239	104	79	79
query85	802	411	308	308
query86	348	296	307	296
query87	4237	4329	4186	4186
query88	5017	2408	2413	2408
query89	402	330	299	299
query90	1874	190	188	188
query91	137	147	108	108
query92	65	57	55	55
query93	1561	876	551	551
query94	697	420	274	274
query95	354	283	281	281
query96	494	617	281	281
query97	3162	3311	3171	3171
query98	220	214	204	204
query99	1495	1430	1309	1309
Total cold run time: 293419 ms
Total hot run time: 192477 ms

@doris-robot
Copy link

doris-robot commented Sep 28, 2025

ClickBench: Total hot run time: 28.75 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 366687c69af99c4084e016ac28a6d62da5b27fa8, data reload: false

query1	0.03	0.02	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.11	0.11
query5	0.52	0.50	0.52
query6	1.13	0.73	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.49	0.51
query10	0.55	0.54	0.54
query11	0.16	0.10	0.10
query12	0.13	0.11	0.11
query13	0.60	0.60	0.59
query14	0.78	0.80	0.79
query15	0.84	0.85	0.83
query16	0.38	0.39	0.41
query17	1.09	1.05	1.07
query18	0.24	0.21	0.22
query19	1.96	1.87	1.89
query20	0.02	0.01	0.01
query21	15.39	0.90	0.57
query22	0.75	0.67	0.69
query23	15.24	1.42	0.57
query24	3.01	0.78	2.03
query25	0.19	0.27	0.08
query26	0.35	0.15	0.14
query27	0.05	0.04	0.04
query28	13.38	0.99	0.43
query29	12.56	3.93	3.22
query30	0.26	0.09	0.06
query31	2.82	0.59	0.37
query32	3.22	0.53	0.47
query33	3.00	3.03	3.08
query34	16.77	5.11	4.53
query35	4.58	4.54	4.59
query36	0.64	0.49	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 103.7 s
Total hot run time: 28.75 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Sep 28, 2025

FE UT Coverage Report

Increment line coverage 0.00% (0/4) 🎉
Increment coverage report
Complete coverage report

@morrySnow morrySnow merged commit 2b22990 into branch-3.1 Sep 29, 2025
23 checks passed
@github-actions github-actions bot deleted the auto-pick-56551-branch-3.1 branch September 29, 2025 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @morrySnow @dataroaring @sollhui