-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Bug](topn) variant column read in topn may coredump #52573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
TPC-H: Total hot run time: 34216 ms |
csun5285
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by anyone and no changes requested. |
TPC-DS: Total hot run time: 185258 ms |
eldenmoon
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
ClickBench: Total hot run time: 29.55 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
### What problem does this PR solve? cherry pick #52573 Problem Summary:
some Regression test coredump when segment cache miss: ``` thrift error, reason=No more data to read.thrift error, reason=No more data to read.thrift error, reason=No more data to read.terminate called after throwing an instance of 'doris::Exception' what(): [E3] Method insert_many_dict_data is not supported for variant *** Query id: 0-0 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1749816938 (unix time) try "date -d @1749816938" if you are using GNU date *** *** Current BE git commitID: a8f84d2 *** *** SIGABRT unknown detail explain (@0x3e9003b76c0) received by PID 3897024 (TID 3898413 OR 0x7c0e01b00700) from PID 3897024; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/tengjianping/doris-master/be/src/common/signal_handler.h:421 1# 0x00007F013A4705B0 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# __GI_abort in /lib64/libc.so.6 4# __gnu_cxx::__verbose_terminate_handler() [clone .cold] in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 5# __cxxabiv1::__terminate(void (*)()) in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 6# 0x00005640CA3EFC71 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 7# 0x00005640CA3EFDC4 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 8# doris::vectorized::IColumn::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column.h:247 9# doris::vectorized::ColumnNullable::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column_nullable.h:242 10# doris::segment_v2::BinaryDictPageDecoder::read_by_rowids(unsigned int const*, unsigned long, unsigned long*, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/binary_dict_page.cpp:325 11# doris::segment_v2::FileColumnIterator::read_by_rowids(unsigned int const*, unsigned long, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/column_reader.cpp:1368 12# doris::segment_v2::Segment::seek_and_read_by_rowid(doris::TabletSchema const&, doris::SlotDescriptor*, unsigned int, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&, doris::OlapReaderStatistics&, std::unique_ptr<doris::segment_v2::ColumnIterator, std::default_delete<doris::segment_v2::ColumnIterator> >&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/segment.cpp:1163 13# doris::RowIdStorageReader::read_doris_format_row(std::shared_ptr<doris::IdFileMap> const&, std::shared_ptr<doris::FileMapping> const&, long, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TabletSchema const&, doris::RowStoreReadStruct&, doris::OlapReaderStatistics&, long*, long*, long*, long*, std::unordered_map<doris::IteratorKey, doris::IteratorItem, doris::HashOfIteratorKey, std::equal_to<doris::IteratorKey>, std::allocator<std::pair<doris::IteratorKey const, doris::IteratorItem> > >&, doris::vectorized::Block&) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:816 14# doris::RowIdStorageReader::read_batch_doris_format_row(doris::PRequestBlockDesc const&, std::shared_ptr<doris::IdFileMap>, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TUniqueId const&, doris::vectorized::Block&, doris::OlapReaderStatistics&, long*, long*, long*, long*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:624 15# doris::RowIdStorageReader::read_by_rowids(doris::PMultiGetRequestV2 const&, doris::PMultiGetResponseV2*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:538 16# doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/service/internal_service.cpp:2124 17# void std::__invoke_impl<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 18# std::enable_if<is_invocable_r_v<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 19# std::_Function_handler<void (), doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 20# std::function<void ()>::operator()() const at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 21# doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::{lambda()apache#1}::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/vec/exec/scan/scanner_scheduler.h:134 ```
some Regression test coredump when segment cache miss: ``` thrift error, reason=No more data to read.thrift error, reason=No more data to read.thrift error, reason=No more data to read.terminate called after throwing an instance of 'doris::Exception' what(): [E3] Method insert_many_dict_data is not supported for variant *** Query id: 0-0 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1749816938 (unix time) try "date -d @1749816938" if you are using GNU date *** *** Current BE git commitID: a8f84d2 *** *** SIGABRT unknown detail explain (@0x3e9003b76c0) received by PID 3897024 (TID 3898413 OR 0x7c0e01b00700) from PID 3897024; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/tengjianping/doris-master/be/src/common/signal_handler.h:421 1# 0x00007F013A4705B0 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# __GI_abort in /lib64/libc.so.6 4# __gnu_cxx::__verbose_terminate_handler() [clone .cold] in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 5# __cxxabiv1::__terminate(void (*)()) in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 6# 0x00005640CA3EFC71 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 7# 0x00005640CA3EFDC4 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 8# doris::vectorized::IColumn::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column.h:247 9# doris::vectorized::ColumnNullable::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column_nullable.h:242 10# doris::segment_v2::BinaryDictPageDecoder::read_by_rowids(unsigned int const*, unsigned long, unsigned long*, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/binary_dict_page.cpp:325 11# doris::segment_v2::FileColumnIterator::read_by_rowids(unsigned int const*, unsigned long, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/column_reader.cpp:1368 12# doris::segment_v2::Segment::seek_and_read_by_rowid(doris::TabletSchema const&, doris::SlotDescriptor*, unsigned int, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&, doris::OlapReaderStatistics&, std::unique_ptr<doris::segment_v2::ColumnIterator, std::default_delete<doris::segment_v2::ColumnIterator> >&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/segment.cpp:1163 13# doris::RowIdStorageReader::read_doris_format_row(std::shared_ptr<doris::IdFileMap> const&, std::shared_ptr<doris::FileMapping> const&, long, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TabletSchema const&, doris::RowStoreReadStruct&, doris::OlapReaderStatistics&, long*, long*, long*, long*, std::unordered_map<doris::IteratorKey, doris::IteratorItem, doris::HashOfIteratorKey, std::equal_to<doris::IteratorKey>, std::allocator<std::pair<doris::IteratorKey const, doris::IteratorItem> > >&, doris::vectorized::Block&) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:816 14# doris::RowIdStorageReader::read_batch_doris_format_row(doris::PRequestBlockDesc const&, std::shared_ptr<doris::IdFileMap>, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TUniqueId const&, doris::vectorized::Block&, doris::OlapReaderStatistics&, long*, long*, long*, long*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:624 15# doris::RowIdStorageReader::read_by_rowids(doris::PMultiGetRequestV2 const&, doris::PMultiGetResponseV2*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:538 16# doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/service/internal_service.cpp:2124 17# void std::__invoke_impl<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 18# std::enable_if<is_invocable_r_v<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 19# std::_Function_handler<void (), doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 20# std::function<void ()>::operator()() const at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 21# doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::{lambda()apache#1}::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/vec/exec/scan/scanner_scheduler.h:134 ```
some Regression test coredump when segment cache miss: ``` thrift error, reason=No more data to read.thrift error, reason=No more data to read.thrift error, reason=No more data to read.terminate called after throwing an instance of 'doris::Exception' what(): [E3] Method insert_many_dict_data is not supported for variant *** Query id: 0-0 *** *** is nereids: 1 *** *** tablet id: 0 *** *** Aborted at 1749816938 (unix time) try "date -d @1749816938" if you are using GNU date *** *** Current BE git commitID: a8f84d2 *** *** SIGABRT unknown detail explain (@0x3e9003b76c0) received by PID 3897024 (TID 3898413 OR 0x7c0e01b00700) from PID 3897024; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /mnt/disk2/tengjianping/doris-master/be/src/common/signal_handler.h:421 1# 0x00007F013A4705B0 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# __GI_abort in /lib64/libc.so.6 4# __gnu_cxx::__verbose_terminate_handler() [clone .cold] in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 5# __cxxabiv1::__terminate(void (*)()) in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 6# 0x00005640CA3EFC71 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 7# 0x00005640CA3EFDC4 in /mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be 8# doris::vectorized::IColumn::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column.h:247 9# doris::vectorized::ColumnNullable::insert_many_dict_data(int const*, unsigned long, doris::StringRef const*, unsigned long, unsigned int) at /mnt/disk2/tengjianping/doris-master/be/src/vec/columns/column_nullable.h:242 10# doris::segment_v2::BinaryDictPageDecoder::read_by_rowids(unsigned int const*, unsigned long, unsigned long*, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/binary_dict_page.cpp:325 11# doris::segment_v2::FileColumnIterator::read_by_rowids(unsigned int const*, unsigned long, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/column_reader.cpp:1368 12# doris::segment_v2::Segment::seek_and_read_by_rowid(doris::TabletSchema const&, doris::SlotDescriptor*, unsigned int, doris::COW<doris::vectorized::IColumn>::mutable_ptr<doris::vectorized::IColumn>&, doris::OlapReaderStatistics&, std::unique_ptr<doris::segment_v2::ColumnIterator, std::default_delete<doris::segment_v2::ColumnIterator> >&) at /mnt/disk2/tengjianping/doris-master/be/src/olap/rowset/segment_v2/segment.cpp:1163 13# doris::RowIdStorageReader::read_doris_format_row(std::shared_ptr<doris::IdFileMap> const&, std::shared_ptr<doris::FileMapping> const&, long, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TabletSchema const&, doris::RowStoreReadStruct&, doris::OlapReaderStatistics&, long*, long*, long*, long*, std::unordered_map<doris::IteratorKey, doris::IteratorItem, doris::HashOfIteratorKey, std::equal_to<doris::IteratorKey>, std::allocator<std::pair<doris::IteratorKey const, doris::IteratorItem> > >&, doris::vectorized::Block&) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:816 14# doris::RowIdStorageReader::read_batch_doris_format_row(doris::PRequestBlockDesc const&, std::shared_ptr<doris::IdFileMap>, std::vector<doris::SlotDescriptor, std::allocator<doris::SlotDescriptor> >&, doris::TUniqueId const&, doris::vectorized::Block&, doris::OlapReaderStatistics&, long*, long*, long*, long*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:624 15# doris::RowIdStorageReader::read_by_rowids(doris::PMultiGetRequestV2 const&, doris::PMultiGetResponseV2*) at /mnt/disk2/tengjianping/doris-master/be/src/exec/rowid_fetcher.cpp:538 16# doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/service/internal_service.cpp:2124 17# void std::__invoke_impl<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(std::__invoke_other, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61 18# std::enable_if<is_invocable_r_v<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&>(doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:117 19# std::_Function_handler<void (), doris::PInternalService::multiget_data_v2(google::protobuf::RpcController*, doris::PMultiGetRequestV2 const*, doris::PMultiGetResponseV2*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:290 20# std::function<void ()>::operator()() const at /mnt/disk2/tengjianping/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_function.h:591 21# doris::vectorized::SimplifiedScanScheduler::submit_scan_task(doris::vectorized::SimplifiedScanTask)::{lambda()apache#1}::operator()() const at /mnt/disk2/tengjianping/doris-master/be/src/vec/exec/scan/scanner_scheduler.h:134 ```
### What problem does this PR solve? cherry pick #52573 Problem Summary:
What problem does this PR solve?
some Regression test coredump when segment cache miss:
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)