[Bug] select array with substring core #17491
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
master
What's Wrong?
be core stack
doris_be: /incubator-doris/be/src/vec/common/pod_array.h:343: const T& doris::vectorized::PODArray<T, initial_bytes, TAllocator, pad_right_, pad_left_>::operator[](ssize_t) const [with T = long unsigned int; long unsigned int initial_bytes = 4096; TAllocator = Allocator<false>; long unsigned int pad_right_ = 15; long unsigned int pad_left_ = 16; ssize_t = long int]: Assertion `(n >= (static_cast<ssize_t>(pad_left_) ? -1 : 0)) && (n <= static_cast<ssize_t>(this->size()))' failed.
--Type <RET> for more, q to quit, c to continue without paging--b
Thread 389 "_scanner_scan" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffe8ebfb700 (LWP 142759)]
0x00007ffff708537f in raise () from /lib64/libc.so.6
(gdb) bt
#0 0x00007ffff708537f in raise () from /lib64/libc.so.6
#1 0x00007ffff706fdb5 in abort () from /lib64/libc.so.6
#2 0x00007ffff706fc89 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3 0x00007ffff707da76 in __assert_fail () from /lib64/libc.so.6
#4 0x0000555567187dcf in doris::vectorized::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>::operator[] (this=0x7fffe041d190, n=1)
at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/common/pod_array.h:343
#5 0x000055556a1801b3 in doris::vectorized::ColumnArray::size_at (i=1, this=0x7fffe041ca60) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_array.h:206
#6 doris::vectorized::ColumnArray::filter_generic (this=0x7fffe041ca60, filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_array.cpp:634
#7 0x000055556a17e9bd in doris::vectorized::ColumnArray::filter (this=0x7fffe041ca60, filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_array.cpp:417
#8 0x000055556a18079f in doris::vectorized::ColumnArray::filter_nullable (this=0x7fffe041c9c0, filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_array.cpp:673
#9 0x000055556a17e9a8 in doris::vectorized::ColumnArray::filter (this=0x7fffe041c9c0, filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_array.cpp:415
#10 0x000055556a1eb218 in doris::vectorized::ColumnNullable::filter (this=0x7fffe040b670, filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/columns/column_nullable.cpp:313
#11 0x000055556a36a3c4 in doris::vectorized::Block::filter_block_internal (block=0x7ffec0e43260, columns_to_filter=..., filter=...) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/core/block.cpp:672
#12 0x000055556a36ab9c in doris::vectorized::Block::filter_block (block=0x7ffec0e43260, columns_to_filter=..., filter_column_id=6, column_to_keep=2) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/core/block.cpp:732
#13 0x000055556a36ae3e in doris::vectorized::Block::filter_block (block=0x7ffec0e43260, filter_column_id=6, column_to_keep=2) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/core/block.cpp:757
#14 0x000055556bf56794 in doris::vectorized::VExprContext::filter_block (vexpr_ctx=0x7ffed42ef780, block=0x7ffec0e43260, column_to_keep=2) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/exprs/vexpr_context.cpp:112
#15 0x000055557007f8ff in doris::vectorized::VScanner::_filter_output_block (this=0x7ffec0e4d000, block=0x7ffec0e43260) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/exec/scan/vscanner.cpp:95
#16 0x000055557007f608 in doris::vectorized::VScanner::get_block (this=0x7ffec0e4d000, state=0x7ffee3494700, block=0x7ffec0e43260, eof=0x7ffe8ebf31b5)
at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/exec/scan/vscanner.cpp:72
#17 0x0000555570068468 in doris::vectorized::ScannerScheduler::_scanner_scan (this=0x7fff854598c0, scheduler=0x7fff854598c0, ctx=0x7ffed426cf00, scanner=0x7ffec0e4d000)
at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:306
#18 0x00005555700678ba in operator() (__closure=0x7fffe041c920) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/vec/exec/scan/scanner_scheduler.cpp:184
#19 0x0000555570069e8c in std::__invoke_impl<void, doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()>::<lambda()>&>(std::__invoke_other, struct {...} &) (__f=...)
at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/invoke.h:61
#20 0x0000555570069c7a in std::__invoke_r<void, doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()>::<lambda()>&>(struct {...} &) (__fn=...)
at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/invoke.h:111
#21 0x0000555570069865 in std::_Function_handler<void(), doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()>::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/std_function.h:291
#22 0x00005555670c1180 in std::function<void ()>::operator()() const (this=0x7ffe8ebf3438) at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/std_function.h:560
#23 0x000055557006cdbb in doris::PriorityWorkStealingThreadPool::work_thread (this=0x7ffff5313a80, thread_id=4) at /mnt/disk2/luen/develop/workspace/incubator-doris/be/src/util/priority_work_stealing_thread_pool.hpp:135
#24 0x0000555570070a0f in std::__invoke_impl<void, void (doris::PriorityWorkStealingThreadPool::* const&)(int), doris::PriorityWorkStealingThreadPool*&, int&> (__f=
@0x7fff85466388: (void (doris::PriorityWorkStealingThreadPool::*)(doris::PriorityWorkStealingThreadPool * const, int)) 0x55557006cbb0 <doris::PriorityWorkStealingThreadPool::work_thread(int)>, __t=@0x7fff854663a0: 0x7ffff5313a80)
at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/invoke.h:74
#25 0x0000555570070979 in std::__invoke<void (doris::PriorityWorkStealingThreadPool::* const&)(int), doris::PriorityWorkStealingThreadPool*&, int&> (__fn=
@0x7fff85466388: (void (doris::PriorityWorkStealingThreadPool::*)(doris::PriorityWorkStealingThreadPool * const, int)) 0x55557006cbb0 <doris::PriorityWorkStealingThreadPool::work_thread(int)>)
at /mnt/disk2/luen/develop/tools/ldb_toolchain/include/c++/11/bits/invoke.h:96
#26 0x0000555570070915 in std::_Mem_fn_base<void (doris::PriorityWorkStealingThreadPool::*)(int), true>::operator()<doris::PriorityWorkStealingThreadPool*&, int&> (this=0x7fff85466388)
What You Expected?
Query normal.
How to Reproduce?
- start es docker
cd docker/thirdparties/ && ./run-thirdparties-docker.sh -c es
- open es external regression test
vim regression-test/conf/regression-conf.groovy
enableEsTest=true
- run es p0, regression will failed.
./run-regression-test.sh --run -s test_es_query- create olap table
CREATE TABLE `test1` (
`id` int(11) NULL,
`c_bool` array<boolean> NULL COMMENT 'Elasticsearch type is boolean',
`c_byte` array<tinyint(4)> NULL COMMENT 'Elasticsearch type is byte',
`c_date` array<datev2> NULL COMMENT 'Elasticsearch type is date, format is yyyy-MM-dd',
`c_datetime` array<datetimev2(0)> NULL COMMENT 'Elasticsearch type is date, format is yyyy-MM-dd HH:mm:ss',
`c_double` array<double> NULL COMMENT 'Elasticsearch type is double',
`c_float` array<float> NULL COMMENT 'Elasticsearch type is float',
`c_half_float` array<float> NULL COMMENT 'Elasticsearch type is half_float',
`c_integer` array<int(11)> NULL COMMENT 'Elasticsearch type is integer',
`c_ip` array<text> NULL COMMENT 'Elasticsearch type is ip',
`c_keyword` array<text> NULL COMMENT 'Elasticsearch type is keyword',
`c_long` array<bigint(20)> NULL COMMENT 'Elasticsearch type is long',
`c_person` array<text> NULL COMMENT 'Elasticsearch no type',
`c_scaled_float` array<double> NULL COMMENT 'Elasticsearch type is scaled_float',
`c_short` array<smallint(6)> NULL COMMENT 'Elasticsearch type is short',
`c_text` array<text> NULL COMMENT 'Elasticsearch type is text',
`c_unsigned_long` array<largeint(40)> NULL COMMENT 'Elasticsearch type is unsigned_long',
`test1` text NULL COMMENT 'Elasticsearch type is keyword',
`test2` text NULL COMMENT 'Elasticsearch type is text',
`test3` double NULL COMMENT 'Elasticsearch type is double',
`test4` datetimev2(0) NULL COMMENT 'Elasticsearch type is date, no format',
`test5` datetimev2(0) NULL COMMENT 'Elasticsearch type is date, format is yyyy-MM-dd HH:mm:ss',
`test6` bigint(20) NULL COMMENT 'Elasticsearch type is date, format is epoch_millis',
`test7` text NULL COMMENT 'Elasticsearch type is date, format is yyyy-MM-dd HH:mm:ss not support, use String type'
) ENGINE=OLAP
DUPLICATE KEY(`id`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`id`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false"
);- insert values
insert into test1 select 1 as id, * from es7.default_db.test2;- select core
select * from test1;
+------+--------------+----------------+--------------------------+--------------------------------------------+--------------+--------------------+--------------+--------------------------------+------------------------------+-----------------+---------------+---------------------------------------------------------+----------------+------------------------+-----------------+-----------------+---------+-----------+-------+---------------------+---------------------+---------------+---------------------+
| id | c_bool | c_byte | c_date | c_datetime | c_double | c_float | c_half_float | c_integer | c_ip | c_keyword | c_long | c_person | c_scaled_float | c_short | c_text | c_unsigned_long | test1 | test2 | test3 | test4 | test5 | test6 | test7 |
+------+--------------+----------------+--------------------------+--------------------------------------------+--------------+--------------------+--------------+--------------------------------+------------------------------+-----------------+---------------+---------------------------------------------------------+----------------+------------------------+-----------------+-----------------+---------+-----------+-------+---------------------+---------------------+---------------+---------------------+
| 1 | [1, 0, 1, 1] | [1, -2, -3, 4] | [2020-01-01, 2020-01-02] | [2020-01-01 12:00:00, 2020-01-02 13:01:01] | [1, 2, 3, 4] | [1, 1.1, 1.2, 1.3] | [1, 2, 3, 4] | [32768, 32769, -32769, -32770] | ['192.168.0.1', '127.0.0.1'] | ['a', 'b', 'c'] | [-1, 0, 1, 2] | ['{"name":"Andy","age":18}', '{"name":"Tim","age":28}'] | [1, 2, 3, 4] | [128, 129, -129, -130] | ['d', 'e', 'f'] | [0, 1, 2, 3] | string3 | text3_4*5 | 5 | 2022-08-08 00:00:00 | 2022-08-10 12:10:10 | 1660018210000 | 2022-08-10 12:10:10 |
| 1 | [1, 0, 1, 1] | [1, -2, -3, 4] | [2020-01-01, 2020-01-02] | [2020-01-01 12:00:00, 2020-01-02 13:01:01] | [1, 2, 3, 4] | [1, 1.1, 1.2, 1.3] | [1, 2, 3, 4] | [32768, 32769, -32769, -32770] | ['192.168.0.1', '127.0.0.1'] | ['a', 'b', 'c'] | [-1, 0, 1, 2] | ['{"name":"Andy","age":18}', '{"name":"Tim","age":28}'] | [1, 2, 3, 4] | [128, 129, -129, -130] | ['d', 'e', 'f'] | [0, 1, 2, 3] | string2 | text2 | 4 | 2022-08-08 00:00:00 | 2022-08-09 12:10:10 | 1660018210000 | 2022-08-09 12:10:10 |
| 1 | [1, 0, 1, 1] | [1, -2, -3, 4] | [2020-01-01, 2020-01-02] | [2020-01-01 12:00:00, 2020-01-02 13:01:01] | [1, 2, 3, 4] | [1, 1.1, 1.2, 1.3] | [1, 2, 3, 4] | [32768, 32769, -32769, -32770] | ['192.168.0.1', '127.0.0.1'] | ['a', 'b', 'c'] | [-1, 0, 1, 2] | ['{"name":"Andy","age":18}', '{"name":"Tim","age":28}'] | [1, 2, 3, 4] | [128, 129, -129, -130] | ['d', 'e', 'f'] | [0, 1, 2, 3] | string1 | text#1 | 3.14 | 2022-08-08 00:00:00 | 2022-08-08 12:10:10 | 1659931810000 | 1659931810000 |
+------+--------------+----------------+--------------------------+--------------------------------------------+--------------+--------------------+--------------+--------------------------------+------------------------------+-----------------+---------------+---------------------------------------------------------+----------------+------------------------+-----------------+-----------------+---------+-----------+-------+---------------------+---------------------+---------------+---------------------+
mysql> select c_unsigned_long from test1 where test2 = 'text2';
+-----------------+
| c_unsigned_long |
+-----------------+
| [0, 1, 2, 3] |
+-----------------+
1 row in set (0.03 sec)
select c_unsigned_long from test1 where substring(test2, 2) = 'ext2';
be will core Anything Else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct