Skip to content

[fix](planner) Fix incosistency between groupby expression and output of aggregation node #17583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
morningman merged 1 commit into from
Mar 9, 2023
Merged

[fix](planner) Fix incosistency between groupby expression and output of aggregation node #17583

morningman merged 1 commit into from
Mar 9, 2023

Conversation

@Kikyou1997
Copy link
Contributor

@Kikyou1997 Kikyou1997 commented Mar 8, 2023 •
edited by morrySnow
Loading

Proposed changes

pick from #17438

Inconsistent nullability between group by expression and its corresponding output slot would cause BE core as follows:

start time: Wed Mar  8 17:41:58 CST 2023
==172690==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fd94e7b4880; bottom 0x7fda9d195000; size: 0xfffffffeb161f880 (-5613946752)
False positive error reports may follow
For details see https://github.com/google/sanitizers/issues/189
F0308 18:00:08.342861 174092 vaggregation_node.cpp:325] Check failed: nullable_output 
*** Check failure stack trace: ***
    @         0x22c351cd  google::LogMessage::Fail()
    @         0x22c37709  google::LogMessage::SendToLog()
    @         0x22c34d36  google::LogMessage::Flush()
    @         0x22c37d79  google::LogMessageFatal::~LogMessageFatal()
    @         0x11a119f3  doris::vectorized::AggregationNode::prepare()
    @          0xd624efe  doris::ExecNode::prepare()
    @         0x12004379  doris::vectorized::HashJoinNode::prepare()
    @          0xd624efe  doris::ExecNode::prepare()
    @         0x12004379  doris::vectorized::HashJoinNode::prepare()
    @          0xd624efe  doris::ExecNode::prepare()
    @         0x11a0f9a9  doris::vectorized::AggregationNode::prepare()
    @          0xd624efe  doris::ExecNode::prepare()
    @         0x11c3e01e  doris::vectorized::VSortNode::prepare()
    @          0xe257cc5  doris::PlanFragmentExecutor::prepare()
    @          0xe1cf418  doris::FragmentExecState::prepare()
    @          0xe1da360  doris::FragmentMgr::exec_plan_fragment()
    @          0xe1d60aa  doris::FragmentMgr::exec_plan_fragment()
    @          0xe83be52  doris::PInternalServiceImpl::_exec_plan_fragment()
    @          0xe83b28f  doris::PInternalServiceImpl::exec_plan_fragment()
    @          0xe83c215  doris::PInternalServiceImpl::exec_plan_fragment_prepare()
    @          0xf6fac94  doris::PBackendService::CallMethod()
    @         0x234c34b5  brpc::policy::ProcessHttpRequest()
    @         0x23472ce7  brpc::ProcessInputMessage()
    @         0x23473c01  brpc::InputMessenger::OnNewMessages()
    @         0x23596c6e  brpc::Socket::ProcessEvent()
    @         0x2340645f  bthread::TaskGroup::task_runner()
    @         0x234038d1  bthread_make_fcontext
*** Query id: e01df43af50c4a87-96f6bbc5036a74af ***
*** Aborted at 1678269608 (unix time) try "date -d @1678269608" if you are using GNU date ***
*** Current BE git commitID: 4614c2c5a ***
*** SIGABRT unkown detail explain (@0x2a292) received by PID 172690 (TID 0x7fd94e7bd640) from PID 172690; stack trace: ***
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/common/signal_handler.h:420
 1# 0x00007FDBAD91A040 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise in /lib/x86_64-linux-gnu/libc.so.6
 3# abort in /lib/x86_64-linux-gnu/libc.so.6
 4# 0x0000000022C3FBB9 in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
 5# 0x0000000022C351CD in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
 9# doris::vectorized::AggregationNode::prepare(doris::RuntimeState*) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
10# doris::ExecNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/exec/exec_node.cpp:239
11# doris::vectorized::HashJoinNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/vec/exec/join/vhash_join_node.cpp:350
12# doris::ExecNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/exec/exec_node.cpp:239
13# doris::vectorized::HashJoinNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/vec/exec/join/vhash_join_node.cpp:350
14# doris::ExecNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/exec/exec_node.cpp:239
15# doris::vectorized::AggregationNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/vec/exec/vaggregation_node.cpp:294
16# doris::ExecNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/exec/exec_node.cpp:239
17# doris::vectorized::VSortNode::prepare(doris::RuntimeState*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/vec/exec/vsort_node.cpp:70
18# doris::PlanFragmentExecutor::prepare(doris::TExecPlanFragmentParams const&, doris::QueryFragmentsCtx*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/runtime/plan_fragment_executor.cpp:162
19# doris::FragmentExecState::prepare(doris::TExecPlanFragmentParams const&) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
20# doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::PlanFragmentExecutor*)>) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/runtime/fragment_mgr.cpp:706
21# doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/runtime/fragment_mgr.cpp:555
22# doris::PInternalServiceImpl::_exec_plan_fragment(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, doris::PFragmentRequestVersion, bool) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/service/internal_service.cpp:372
23# doris::PInternalServiceImpl::exec_plan_fragment(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/service/internal_service.cpp:195
24# doris::PInternalServiceImpl::exec_plan_fragment_prepare(google::protobuf::RpcController*, doris::PExecPlanFragmentRequest const*, doris::PExecPlanFragmentResult*, google::protobuf::Closure*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/be/src/service/internal_service.cpp:207
25# doris::PBackendService::CallMethod(google::protobuf::MethodDescriptor const*, google::protobuf::RpcController*, google::protobuf::Message const*, google::protobuf::Message*, google::protobuf::Closure*) at /mnt/hdd01/repo_center/doris_branch-1.2-lts/doris/gensrc/build/gen_cpp/internal_service.pb.cc:24578
26# brpc::policy::ProcessHttpRequest(brpc::InputMessageBase*) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
27# brpc::ProcessInputMessage(void*) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
28# brpc::InputMessenger::OnNewMessages(brpc::Socket*) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
29# brpc::Socket::ProcessEvent(void*) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
30# bthread::TaskGroup::task_runner(long) in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be
31# bthread_make_fcontext in /mnt/hdd01/branch-1.2-lts/VEC_ASAN/be/lib/doris_be

Fix it by setting the nullability of slot by the groupby expression's nullability in the finalize phase.

Checklist(Required)

  • Does it affect the original behavior
  • Has unit tests been added
  • Has document been added or modified
  • Does it need to update dependencies
  • Is this PR support rollback (If NO, please explain WHY)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Mar 8, 2023
@Kikyou1997 Kikyou1997 changed the title fix [fix](planner) Fix incosistency between groupby expression and output of aggregation node Mar 8, 2023
@Kikyou1997
Copy link
Contributor Author

Kikyou1997 commented Mar 8, 2023

run buildall

@morningman morningman merged commit 5259892 into apache:branch-1.2-lts Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Kikyou1997 @morningman