Skip to content

[branch-2.0](thrift) fix TLoadTxnBeginRequest backend_id's field no #39821

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
dataroaring merged 1 commit into from
Aug 23, 2024
Merged

[branch-2.0](thrift) fix TLoadTxnBeginRequest backend_id's field no #39821

dataroaring merged 1 commit into from
Aug 23, 2024

Conversation

@yujun777
Copy link
Contributor

@yujun777 yujun777 commented Aug 23, 2024 •
edited
Loading

Make backend_id's field no the same with master branch.

For upgrading, change backend_id's field no is safe, because old fe can torrent with TLoadTxnBeginRequest not setting backend id.

backend_id was introduce by #39317

@doris-robot
Copy link

doris-robot commented Aug 23, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yujun777
Copy link
Contributor Author

yujun777 commented Aug 23, 2024

run buildall

@github-actions
Copy link
Contributor

github-actions bot commented Aug 23, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

doris-robot commented Aug 23, 2024

TPC-H: Total hot run time: 49712 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit faffed4c55c72215dd9823898e5a382b36cb3460, data reload: false

------ Round 1 ----------------------------------
q1	17570	4435	4336	4336
q2	2036	152	145	145
q3	10425	1913	1923	1913
q4	10316	1280	1333	1280
q5	8730	3894	3861	3861
q6	225	125	122	122
q7	2068	1611	1603	1603
q8	9377	2742	2729	2729
q9	10646	10305	10265	10265
q10	8622	3517	3494	3494
q11	416	251	251	251
q12	477	297	303	297
q13	18319	3957	4024	3957
q14	357	354	326	326
q15	508	451	470	451
q16	678	577	589	577
q17	1147	954	951	951
q18	7352	6756	6842	6756
q19	1723	1600	1550	1550
q20	527	294	323	294
q21	4453	4151	4188	4151
q22	495	410	403	403
Total cold run time: 116467 ms
Total hot run time: 49712 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4347	4311	4299	4299
q2	322	227	226	226
q3	4180	4134	4157	4134
q4	2757	2747	2755	2747
q5	7195	7081	7141	7081
q6	238	121	119	119
q7	3280	2870	2823	2823
q8	4369	4431	4462	4431
q9	16927	16835	16758	16758
q10	4228	4270	4262	4262
q11	754	686	705	686
q12	1013	873	848	848
q13	6915	3692	3711	3692
q14	458	424	430	424
q15	499	458	459	458
q16	729	679	692	679
q17	3907	3856	3891	3856
q18	8855	8739	8789	8739
q19	1742	1717	1647	1647
q20	2364	2143	2088	2088
q21	8498	8547	8546	8546
q22	1021	970	983	970
Total cold run time: 84598 ms
Total hot run time: 79513 ms

@doris-robot
Copy link

doris-robot commented Aug 23, 2024

TPC-DS: Total hot run time: 201375 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit faffed4c55c72215dd9823898e5a382b36cb3460, data reload: false

query1	933	424	379	379
query2	6527	2191	2141	2141
query3	6919	205	201	201
query4	20124	17954	17902	17902
query5	19729	6502	6576	6502
query6	283	224	239	224
query7	4162	295	295	295
query8	265	250	237	237
query9	3110	2690	2621	2621
query10	415	300	312	300
query11	11466	10741	10833	10741
query12	130	74	79	74
query13	5599	654	636	636
query14	17737	13710	13081	13081
query15	363	213	231	213
query16	6469	281	260	260
query17	1701	1472	872	872
query18	2330	426	412	412
query19	199	150	159	150
query20	76	79	79	79
query21	190	95	93	93
query22	5290	5094	5039	5039
query23	32743	32060	31989	31989
query24	6974	6528	6541	6528
query25	510	425	424	424
query26	625	164	163	163
query27	2020	296	295	295
query28	6105	2281	2229	2229
query29	2906	2819	2818	2818
query30	243	168	166	166
query31	927	748	764	748
query32	71	66	58	58
query33	398	258	252	252
query34	854	470	465	465
query35	1103	911	910	910
query36	1282	1194	1236	1194
query37	88	59	63	59
query38	3107	2968	2944	2944
query39	1361	1317	1339	1317
query40	202	92	94	92
query41	39	36	36	36
query42	88	80	85	80
query43	752	551	615	551
query44	1127	714	722	714
query45	241	238	235	235
query46	1229	993	971	971
query47	1821	1772	1775	1772
query48	1007	689	679	679
query49	632	365	359	359
query50	883	653	655	653
query51	4776	4612	4649	4612
query52	93	83	83	83
query53	458	321	311	311
query54	2654	2480	2448	2448
query55	90	79	85	79
query56	223	229	212	212
query57	1216	1217	1009	1009
query58	211	216	199	199
query59	3565	3220	3438	3220
query60	204	192	214	192
query61	101	95	96	95
query62	876	450	517	450
query63	499	339	350	339
query64	2615	1549	1478	1478
query65	3596	3575	3551	3551
query66	810	361	384	361
query67	17611	15465	14920	14920
query68	8996	643	650	643
query69	571	354	348	348
query70	1552	1478	1278	1278
query71	418	302	308	302
query72	6635	3503	3531	3503
query73	739	324	314	314
query74	6356	5955	5959	5955
query75	5225	3754	3692	3692
query76	5538	1140	1209	1140
query77	940	261	248	248
query78	12471	11583	11596	11583
query79	8104	638	635	635
query80	1293	402	396	396
query81	486	233	231	231
query82	1670	102	99	99
query83	174	133	131	131
query84	259	72	70	70
query85	894	329	322	322
query86	338	290	325	290
query87	3270	3049	3017	3017
query88	5097	2290	2276	2276
query89	395	280	290	280
query90	1965	210	210	210
query91	166	123	126	123
query92	59	56	54	54
query93	6110	552	559	552
query94	706	214	204	204
query95	1122	1074	1056	1056
query96	629	329	319	319
query97	6464	6345	6435	6345
query98	196	174	178	174
query99	2908	893	873	873
Total cold run time: 314699 ms
Total hot run time: 201375 ms

@doris-robot
Copy link

doris-robot commented Aug 23, 2024

ClickBench: Total hot run time: 31.2 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit faffed4c55c72215dd9823898e5a382b36cb3460, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.05
query4	1.80	0.07	0.07
query5	0.62	0.52	0.54
query6	1.37	0.61	0.61
query7	0.02	0.01	0.01
query8	0.04	0.03	0.02
query9	0.54	0.47	0.48
query10	0.54	0.53	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.10
query13	0.63	0.62	0.61
query14	0.79	0.78	0.79
query15	0.77	0.76	0.77
query16	0.36	0.37	0.37
query17	1.01	1.03	0.99
query18	0.22	0.27	0.24
query19	1.94	1.88	1.81
query20	0.02	0.02	0.01
query21	15.45	0.60	0.55
query22	2.11	2.46	1.53
query23	17.31	1.02	0.94
query24	4.74	1.41	1.49
query25	0.42	0.04	0.05
query26	0.55	0.16	0.16
query27	0.04	0.04	0.04
query28	7.75	0.75	0.72
query29	12.63	2.37	2.40
query30	0.60	0.60	0.54
query31	2.81	0.38	0.37
query32	3.40	0.50	0.50
query33	3.04	3.11	3.07
query34	15.24	4.80	4.80
query35	4.84	4.86	4.85
query36	1.05	1.00	1.01
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.13	0.15
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 103.6 s
Total hot run time: 31.2 s

@doris-robot
Copy link

doris-robot commented Aug 23, 2024

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit faffed4c55c72215dd9823898e5a382b36cb3460 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@dataroaring dataroaring merged commit 5ec7f46 into apache:branch-2.0 Aug 23, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
@yujun777 @Baymine
[branch-2.0](thrift) fix TLoadTxnBeginRequest backend_id's field no (a…
c45df48 
…pache#39821)

Make backend_id's field no the same with master branch.

For upgrading, change backend_id's field no is safe, because old fe can
torrent with TLoadTxnBeginRequest not setting backend id.

backend_id was introduce by apache#39317
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yujun777 @doris-robot @dataroaring