Skip to content

branch-4.0: [Chore](thrift) add reopen thrift connection on RuntimeQueryStatisticsMgr::report_runtime_query_statistics #56744 #56776

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yiguolei merged 1 commit into from
Oct 11, 2025
Merged

branch-4.0: [Chore](thrift) add reopen thrift connection on RuntimeQueryStatisticsMgr::report_runtime_query_statistics #56744 #56776

yiguolei merged 1 commit into from
Oct 11, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Oct 10, 2025

Cherry-picked from #56744

@BiteTheDDDDt
[Chore](thrift) add reopen thrift connection on RuntimeQueryStatistic…
c5b0fae 
…sMgr::report_runtime_query_statistics (#56744)

try to avoid asan check fail:
```cpp
OpenJDK 64-Bit Server VM warning: Option CriticalJNINatives was deprecated in version 16.0 and will likely be removed in a future release.
Listening for transport dt_socket at address: 8871
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
start BE in local mode
thrift error, reason=write() send(): Broken pipe=================================================================
AddressSanitizer: CHECK failed: asan_thread.cpp:369 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=13525)
    #0 0x557a503468e1 in __asan::CheckUnwind() (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a08e1)
    #1 0x557a50361182 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2bb182)
    #2 0x557a5034a97b in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a497b)
    #3 0x557a502a4a30 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f1fea30)
    #4 0x557a502a67a3 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2007a3)
    #5 0x557a50343878 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29d878)
    #6 0x557a50344f75 in __asan_report_load8 (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29ef75)
    #7 0x557a5039791e in std::__cxx11::basic_string, std::allocator>::_M_create(unsigned long&, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:148:11
    #8 0x557a503984db in void std::__cxx11::basic_string, std::allocator>::_M_construct(char const*, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:291:12
    #9 0x557a503984db in std::__cxx11::basic_string, std::allocator>::basic_string(std::__cxx11::basic_string, std::allocator> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:617:2
    #10 0x557a55b4d77d in std::pair, std::allocator> const, doris::TQueryStatistics>::pair(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_pair.h:312:17
    #11 0x557a55b4d77d in _ZSt12construct_atISt4pairIKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5doris16TQueryStatisticsEEJRKSA_EQaant20is_unbounded_array_vIT_ErqXgsnwcvPvLi0E_SD_pispclsr3stdE7declvalIT0_EEEEEPSD_SG_DpOSF_ /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_construct.h:110:22
    #12 0x557a55b4d77d in void std::allocator_traits, std::allocator> const, doris::TQueryStatistics>>>>::construct, std::allocator> const, doris::TQueryStatistics>, std::pair, std::allocator> const, doris::TQueryStatistics> const&>(std::allocator, std::allocator> const, doris::TQueryStatistics>>>&, std::pair, std::allocator> const, doris::TQueryStatistics>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/alloc_traits.h:676:4
    #13 0x557a55b4d77d in void std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_construct_node, std::allocator> const, doris::TQueryStatistics> const&>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1237:8
    #14 0x557a55b4cfe3 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node::operator(), std::allocator> const, doris::TQueryStatistics> const&>(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h
    #15 0x557a55b4cbe6 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_clone_node, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1287:8
    #16 0x557a55b4c6f6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2546:20
    #17 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3
    #18 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3
    #19 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3
    #20 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3
    #21 0x557a55b4c27a in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1523:6
    #22 0x557a55740d4f in std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2453:18
    #23 0x557a55740d4f in std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_map.h:354:37
    #24 0x557a55740d4f in doris::TReportWorkloadRuntimeStatusParams::__set_query_statistics_map(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /home/zcp/repo_center/doris_branch-4.0/doris/gensrc/build/gen_cpp/FrontendService_types.cpp:4391:30
    #25 0x557a5447e13a in doris::RuntimeQueryStatisticsMgr::report_runtime_query_statistics() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/runtime/runtime_query_statistics_mgr.cpp:403:31
    #26 0x557a504f1192 in doris::Daemon::report_runtime_query_statistics_thread() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/common/daemon.cpp:457:65
    #27 0x557a54bf2eb6 in std::function::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
    #28 0x557a54bf2eb6 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-4.0/doris/be/src/util/thread.cpp:460:5
    #29 0x557a50336d26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f290d26)
    #30 0x7f12a399eac2 in start_thread nptl/pthread_create.c:442:8
    #31 0x7f12a3a3084f in __closefrom_fallback misc/../sysdeps/unix/sysv/linux/closefrom_fallback.c:45:19
172.20.59.34 last coredump sql: last SQL query not found
```
@github-actions github-actions bot requested a review from yiguolei as a code owner October 10, 2025 03:19
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 10, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Oct 10, 2025
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 10, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 10, 2025

ClickBench: Total hot run time: 29.89 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c5b0fae3bb6d708b3df6526238f4aed8003986e3, data reload: true

query1	0.03	0.03	0.04
query2	0.11	0.03	0.04
query3	0.27	0.07	0.07
query4	1.74	0.10	0.11
query5	0.27	0.24	0.25
query6	1.26	0.64	0.63
query7	0.02	0.02	0.02
query8	0.06	0.04	0.03
query9	0.64	0.51	0.51
query10	0.58	0.57	0.57
query11	0.19	0.12	0.11
query12	0.16	0.11	0.12
query13	0.62	0.62	0.61
query14	0.80	0.83	0.81
query15	0.87	0.84	0.87
query16	0.39	0.42	0.40
query17	1.01	1.05	1.06
query18	0.19	0.20	0.20
query19	1.97	1.88	1.89
query20	0.02	0.01	0.01
query21	15.46	0.97	0.57
query22	0.80	1.19	0.74
query23	15.02	1.33	0.57
query24	16.31	0.43	0.18
query25	0.21	0.05	0.05
query26	0.37	0.17	0.12
query27	0.06	0.05	0.05
query28	10.39	0.95	0.92
query29	12.70	3.91	3.25
query30	0.31	0.14	0.11
query31	3.00	0.59	0.39
query32	3.30	0.56	0.48
query33	3.06	3.16	3.06
query34	17.50	5.69	5.13
query35	5.01	5.15	5.15
query36	0.71	0.54	0.53
query37	0.11	0.07	0.06
query38	0.06	0.04	0.03
query39	0.04	0.03	0.03
query40	0.19	0.15	0.16
query41	0.09	0.03	0.02
query42	0.04	0.03	0.02
query43	0.05	0.03	0.04
Total cold run time: 115.99 s
Total hot run time: 29.89 s

@yiguolei yiguolei merged commit 43460b2 into branch-4.0 Oct 11, 2025
22 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hello-stephen @doris-robot @yiguolei @dataroaring @BiteTheDDDDt