[improvement](be report) add be report http #28424

yujun777 · 2023-12-14T12:30:12Z

used for regression test. we can send be report http and no need to wait.

usage:

type	http	regression test suites
report disk	http://beIp:beHttpPort/api/report/disk	be_report_disk(beIp, beHttpPort)
report task	http://beIp:beHttpPort/api/report/task	be_report_task(beIp, beHttpPort)
report tablet	http://beIp:beHttpPort/api/report/tablet	be_report_tablet(beIp, beHttpPort)

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

yujun777 · 2023-12-14T12:30:20Z

run buildall

github-actions

clang-tidy made some suggestions

github-actions · 2023-12-14T12:38:12Z

be/src/olap/storage_engine.cpp

 }

-void StorageEngine::notify_listener(std::string_view name) {
+bool StorageEngine::notify_listener(std::string_view name) {


warning: method 'notify_listener' can be made static [readability-convert-member-functions-to-static]

Suggested change

bool StorageEngine::notify_listener(std::string_view name) {

ner->notify();static

hello-stephen · 2023-12-16T02:10:44Z

run buildall

dataroaring

LGTM

github-actions · 2023-12-17T03:31:04Z

PR approved by at least one committer and no changes requested.

github-actions · 2023-12-17T03:31:06Z

PR approved by anyone and no changes requested.

yujun777 · 2023-12-17T12:21:29Z

run buildall

yujun777 · 2023-12-17T12:41:18Z

run buildall

yujun777 · 2023-12-17T12:54:00Z

run buildall

doris-robot · 2023-12-17T13:46:47Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 82de6fa4ffcc181793468f3b8c2a6db050630dd9, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4745	4469	4506	4469
q2	362	143	160	143
q3	1456	1258	1218	1218
q4	1121	912	927	912
q5	3171	3166	3181	3166
q6	247	126	127	126
q7	1003	485	491	485
q8	2221	2235	2205	2205
q9	6712	6688	6694	6688
q10	3235	3292	3246	3246
q11	327	211	205	205
q12	346	207	203	203
q13	4542	3814	3818	3814
q14	238	209	210	209
q15	568	532	522	522
q16	444	388	381	381
q17	1006	662	560	560
q18	7240	6983	6874	6874
q19	1530	1438	1399	1399
q20	535	315	331	315
q21	3088	2667	2650	2650
q22	344	272	276	272
Total cold run time: 44481 ms
Total hot run time: 40062 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4411	4397	4391	4391
q2	267	165	175	165
q3	3551	3534	3536	3534
q4	2382	2391	2376	2376
q5	5749	5753	5719	5719
q6	243	122	123	122
q7	2381	1895	1886	1886
q8	3540	3541	3532	3532
q9	9049	9039	9000	9000
q10	3914	4007	4016	4007
q11	499	375	385	375
q12	775	592	596	592
q13	4302	3585	3567	3567
q14	283	256	262	256
q15	570	507	528	507
q16	513	460	476	460
q17	1908	1869	1853	1853
q18	8733	8307	8235	8235
q19	1723	1758	1757	1757
q20	2249	1972	1943	1943
q21	6531	6221	6195	6195
q22	515	424	427	424
Total cold run time: 64088 ms
Total hot run time: 60896 ms

yujun777 · 2023-12-18T10:50:29Z

run buildall

doris-robot · 2023-12-18T11:19:27Z

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Tpch sf100 test result on commit 39a18e9f87d9dc2b0bc7b6da2e83432dff6e58f0, data reload: false

run tpch-sf100 query with default conf and session variables
q1	4713	4426	4474	4426
q2	362	153	157	153
q3	1458	1261	1235	1235
q4	1115	881	924	881
q5	3123	3163	3150	3150
q6	244	126	128	126
q7	1000	483	485	483
q8	2205	2224	2205	2205
q9	6686	6667	6667	6667
q10	3206	3269	3286	3269
q11	325	194	190	190
q12	350	209	203	203
q13	4550	3807	3785	3785
q14	249	208	216	208
q15	559	528	523	523
q16	444	382	383	382
q17	1038	638	560	560
q18	7079	7022	6873	6873
q19	1534	1418	1407	1407
q20	555	296	311	296
q21	3080	2660	2685	2660
q22	355	280	285	280
Total cold run time: 44230 ms
Total hot run time: 39962 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	4370	4374	4379	4374
q2	271	164	167	164
q3	3531	3524	3523	3523
q4	2378	2365	2364	2364
q5	5719	5705	5738	5705
q6	240	119	120	119
q7	2409	1885	1860	1860
q8	3519	3511	3514	3511
q9	9016	9029	8992	8992
q10	3912	3971	4014	3971
q11	505	372	402	372
q12	765	603	604	603
q13	4289	3585	3523	3523
q14	285	249	249	249
q15	563	521	528	521
q16	515	463	457	457
q17	1882	1836	1841	1836
q18	8689	8281	8179	8179
q19	1735	1724	1761	1724
q20	2252	1935	1945	1935
q21	6546	6147	6172	6147
q22	494	426	447	426
Total cold run time: 63885 ms
Total hot run time: 60555 ms

doris-robot · 2023-12-18T11:32:16Z

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 43.85 seconds
stream load tsv: 593 seconds loaded 74807831229 Bytes, about 120 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 33 seconds loaded 861443392 Bytes, about 24 MB/s
insert into select: 29.8 seconds inserted 10000000 Rows, about 335K ops/s
storage size: 17224028743 Bytes

dataroaring

LGTM

github-actions · 2023-12-19T02:34:55Z

PR approved by at least one committer and no changes requested.

deardeng

LGTM

dataroaring

LGTM

yujun777 mentioned this pull request Dec 14, 2023

[fix](publish version) fix publish version failed but return ok #28425

Merged

github-actions bot reviewed Dec 14, 2023

View reviewed changes

dataroaring previously approved these changes Dec 17, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 17, 2023

github-actions bot added the reviewed label Dec 17, 2023

add be report http

8066e64

yujun777 force-pushed the add-be-report-http branch from 2fb9dfd to 8066e64 Compare December 17, 2023 12:41

update

82de6fa

yujun777 dismissed dataroaring’s stale review via 82de6fa December 17, 2023 12:53

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Dec 17, 2023

update

39a18e9

dataroaring approved these changes Dec 19, 2023

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Dec 19, 2023

deardeng approved these changes Dec 19, 2023

View reviewed changes

dataroaring approved these changes Dec 19, 2023

View reviewed changes

dataroaring merged commit f9ddf8c into apache:master Dec 19, 2023

yujun777 mentioned this pull request Dec 21, 2023

[branch-2.0](publish version) fix publish version failed but return ok #28425 #28791

Merged

hello-stephen pushed a commit to hello-stephen/doris that referenced this pull request Dec 28, 2023

[improvement](be report) add be report http (apache#28424)

321023a

HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024

[improvement](be report) add be report http (apache#28424)

89d05ee

	bool StorageEngine::notify_listener(std::string_view name) {
	ner->notify();static

[improvement](be report) add be report http #28424

[improvement](be report) add be report http #28424

Uh oh!

Conversation

yujun777 commented Dec 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Further comments

Uh oh!

yujun777 commented Dec 14, 2023

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Dec 14, 2023

Choose a reason for hiding this comment

Uh oh!

hello-stephen commented Dec 16, 2023

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 17, 2023

Uh oh!

github-actions bot commented Dec 17, 2023

Uh oh!

yujun777 commented Dec 17, 2023

Uh oh!

yujun777 commented Dec 17, 2023

Uh oh!

yujun777 commented Dec 17, 2023

Uh oh!

doris-robot commented Dec 17, 2023

Uh oh!

yujun777 commented Dec 18, 2023

Uh oh!

doris-robot commented Dec 18, 2023

Uh oh!

doris-robot commented Dec 18, 2023

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 19, 2023

Uh oh!

deardeng left a comment

Choose a reason for hiding this comment

Uh oh!

dataroaring left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yujun777 commented Dec 14, 2023 •

edited

Loading