Skip to content

20.5 Kafka + JSONAsString = assert fail in PeekableReadBuffer::checkStateCorrect  #11514

Closed
#11599
Closed
20.5 Kafka + JSONAsString = assert fail in PeekableReadBuffer::checkStateCorrect #11514
#11599
Assignees
filimonov
Labels
bugConfirmed user-visible misbehaviour in official releasecomp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).comp-message-queuesMessage queue integrations (Kafka, RabbitMQ, NATS table engines for stream ingestion/egress).
@filimonov

Description

@filimonov
filimonov
opened on Jun 8, 2020
2020.06.08 16:53:02.113533 [ 9 ] {} <Trace> BaseDaemon: Received signal 11
2020.06.08 16:53:02.113791 [ 75 ] {} <Fatal> BaseDaemon: ########################################
2020.06.08 16:53:02.113923 [ 75 ] {} <Fatal> BaseDaemon: (version 20.5.1.1) (from thread 73) (query_id: 11a4a97a-2c5a-4eb4-ad0b-4cbb362ae30e) Received signal Segmentation fault (11).
2020.06.08 16:53:02.114044 [ 75 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
2020.06.08 16:53:02.114120 [ 75 ] {} <Fatal> BaseDaemon: Stack trace: 0x7f658efed8f0 0x7f658efdd39a 0x7f658efdd412 0x157a5608 0x1d0b8566 0x1d0b902c 0x157ac74b 0x157d0f7b 0x1d0b4921 0x1d045f6f 0x1cf4e657 0x1cace454 0x1cacd3fd 0x1bb8fcad 0x1d217e01 0x1cf4e657 0x1d217ab3 0x1cf9f54c 0x1cf9f4c0 0x1cf9f47d 0x1cf9f42d 0x1cf9f3fd 0x1cf9e55e 0x157c7455 0x157c6355 0x1cf9ccf5 0x1cf9d507 0x1cf9ba60
2020.06.08 16:53:02.114388 [ 75 ] {} <Fatal> BaseDaemon: 4. /build/glibc-OTsEL5/glibc-2.27/stdlib/abort.c:107: __GI_abort @ 0x408f0 in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.27.so
2020.06.08 16:53:02.114536 [ 75 ] {} <Fatal> BaseDaemon: 5. /build/glibc-OTsEL5/glibc-2.27/assert/assert.c:89: __assert_fail_base @ 0x3039a in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.27.so
2020.06.08 16:53:02.114780 [ 75 ] {} <Fatal> BaseDaemon: 6. ? @ 0x30412 in /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.27.so
2020.06.08 16:53:02.115076 [ 75 ] {} <Fatal> BaseDaemon: 7. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Common/Exception.cpp:47: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x157a5608 in /usr/bin/clickhouse
2020.06.08 16:53:02.148737 [ 75 ] {} <Fatal> BaseDaemon: 8. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/IO/PeekableReadBuffer.cpp:177: DB::PeekableReadBuffer::checkStateCorrect() const @ 0x1d0b8566 in /usr/bin/clickhouse
2020.06.08 16:53:02.171948 [ 74 ] {} <Fatal> BaseDaemon: 14. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Formats/IRowInputFormat.cpp:64: DB::IRowInputFormat::generate() @ 0x1d045f6f in /usr/bin/clickhouse
2020.06.08 16:53:02.184119 [ 75 ] {} <Fatal> BaseDaemon: 9. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/IO/PeekableReadBuffer.cpp:111: DB::PeekableReadBuffer::nextImpl() @ 0x1d0b902c in /usr/bin/clickhouse
2020.06.08 16:53:02.184527 [ 75 ] {} <Fatal> BaseDaemon: 10. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/IO/ReadBuffer.h:53: DB::ReadBuffer::next() @ 0x157ac74b in /usr/bin/clickhouse
2020.06.08 16:53:02.184699 [ 75 ] {} <Fatal> BaseDaemon: 11. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/IO/ReadBuffer.h:81: DB::skipWhitespaceIfAny(DB::ReadBuffer&) @ 0x157d0f7b in /usr/bin/clickhouse
2020.06.08 16:53:02.221443 [ 75 ] {} <Fatal> BaseDaemon: 12. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Formats/Impl/JSONAsStringRowInputFormat.cpp:103: DB::JSONAsStringRowInputFormat::readRow(std::__1::vector<COW<DB::IColumn>::mutable_ptr<DB::IColumn>, std::__1::allocator<COW<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, DB::RowReadExtension&) @ 0x1d0b4921 in /usr/bin/clickhouse
2020.06.08 16:53:02.258763 [ 75 ] {} <Fatal> BaseDaemon: 13. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Formats/IRowInputFormat.cpp:64: DB::IRowInputFormat::generate() @ 0x1d045f6f in /usr/bin/clickhouse
2020.06.08 16:53:02.274056 [ 74 ] {} <Fatal> BaseDaemon: 15. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/ISource.cpp:48: DB::ISource::work() @ 0x1cf4e657 in /usr/bin/clickhouse
2020.06.08 16:53:02.295283 [ 75 ] {} <Fatal> BaseDaemon: 14. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/ISource.cpp:48: DB::ISource::work() @ 0x1cf4e657 in /usr/bin/clickhouse
2020.06.08 16:53:02.331305 [ 75 ] {} <Fatal> BaseDaemon: 15. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Storages/Kafka/KafkaBlockInputStream.cpp:88: DB::KafkaBlockInputStream::readImpl()::$_0::operator()() const @ 0x1cace454 in /usr/bin/clickhouse
2020.06.08 16:53:02.365384 [ 75 ] {} <Fatal> BaseDaemon: 16. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Storages/Kafka/KafkaBlockInputStream.cpp:128: DB::KafkaBlockInputStream::readImpl() @ 0x1cacd3fd in /usr/bin/clickhouse
2020.06.08 16:53:02.370875 [ 74 ] {} <Fatal> BaseDaemon: 16. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Storages/Kafka/KafkaBlockInputStream.cpp:88: DB::KafkaBlockInputStream::readImpl()::$_0::operator()() const @ 0x1cace454 in /usr/bin/clickhouse
2020.06.08 16:53:02.394171 [ 75 ] {} <Fatal> BaseDaemon: 17. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/DataStreams/IBlockInputStream.cpp:60: DB::IBlockInputStream::read() @ 0x1bb8fcad in /usr/bin/clickhouse
2020.06.08 16:53:02.434780 [ 75 ] {} <Fatal> BaseDaemon: 18. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Sources/SourceFromInputStream.cpp:145: DB::SourceFromInputStream::generate() @ 0x1d217e01 in /usr/bin/clickhouse
2020.06.08 16:53:02.467556 [ 74 ] {} <Fatal> BaseDaemon: 17. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Storages/Kafka/KafkaBlockInputStream.cpp:128: DB::KafkaBlockInputStream::readImpl() @ 0x1cacd3fd in /usr/bin/clickhouse
2020.06.08 16:53:02.471475 [ 75 ] {} <Fatal> BaseDaemon: 19. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/ISource.cpp:48: DB::ISource::work() @ 0x1cf4e657 in /usr/bin/clickhouse
2020.06.08 16:53:02.529883 [ 75 ] {} <Fatal> BaseDaemon: 20. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Sources/SourceFromInputStream.cpp:97: DB::SourceFromInputStream::work() @ 0x1d217ab3 in /usr/bin/clickhouse
2020.06.08 16:53:02.553526 [ 74 ] {} <Fatal> BaseDaemon: 18. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/DataStreams/IBlockInputStream.cpp:60: DB::IBlockInputStream::read() @ 0x1bb8fcad in /usr/bin/clickhouse
2020.06.08 16:53:02.570632 [ 75 ] {} <Fatal> BaseDaemon: 21. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:155: DB::executeJob(DB::IProcessor*) @ 0x1cf9f54c in /usr/bin/clickhouse
2020.06.08 16:53:02.594073 [ 74 ] {} <Fatal> BaseDaemon: 19. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Sources/SourceFromInputStream.cpp:145: DB::SourceFromInputStream::generate() @ 0x1d217e01 in /usr/bin/clickhouse
2020.06.08 16:53:02.610344 [ 75 ] {} <Fatal> BaseDaemon: 22. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:172: DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1::operator()() const @ 0x1cf9f4c0 in /usr/bin/clickhouse
2020.06.08 16:53:02.634033 [ 74 ] {} <Fatal> BaseDaemon: 20. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/ISource.cpp:48: DB::ISource::work() @ 0x1cf4e657 in /usr/bin/clickhouse
2020.06.08 16:53:02.650700 [ 75 ] {} <Fatal> BaseDaemon: 23. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/type_traits:3519: decltype(std::__1::forward<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(fp)()) std::__1::__invoke<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&) @ 0x1cf9f47d in /usr/bin/clickhouse
2020.06.08 16:53:02.676312 [ 74 ] {} <Fatal> BaseDaemon: 21. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Sources/SourceFromInputStream.cpp:97: DB::SourceFromInputStream::work() @ 0x1d217ab3 in /usr/bin/clickhouse
2020.06.08 16:53:02.691960 [ 75 ] {} <Fatal> BaseDaemon: 24. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/__functional_base:349: void std::__1::__invoke_void_return_wrapper<void>::__call<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&) @ 0x1cf9f42d in /usr/bin/clickhouse
2020.06.08 16:53:02.717487 [ 74 ] {} <Fatal> BaseDaemon: 22. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:155: DB::executeJob(DB::IProcessor*) @ 0x1cf9f54c in /usr/bin/clickhouse
2020.06.08 16:53:02.732639 [ 75 ] {} <Fatal> BaseDaemon: 25. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1540: std::__1::__function::__alloc_func<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1, std::__1::allocator<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1>, void ()>::operator()() @ 0x1cf9f3fd in /usr/bin/clickhouse
2020.06.08 16:53:02.757431 [ 74 ] {} <Fatal> BaseDaemon: 23. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:172: DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1::operator()() const @ 0x1cf9f4c0 in /usr/bin/clickhouse
2020.06.08 16:53:02.771716 [ 75 ] {} <Fatal> BaseDaemon: 26. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1714: std::__1::__function::__func<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1, std::__1::allocator<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1>, void ()>::operator()() @ 0x1cf9e55e in /usr/bin/clickhouse
2020.06.08 16:53:02.773112 [ 75 ] {} <Fatal> BaseDaemon: 27. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1867: std::__1::__function::__value_func<void ()>::operator()() const @ 0x157c7455 in /usr/bin/clickhouse
2020.06.08 16:53:02.773316 [ 75 ] {} <Fatal> BaseDaemon: 28. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:2473: std::__1::function<void ()>::operator()() const @ 0x157c6355 in /usr/bin/clickhouse
2020.06.08 16:53:02.796382 [ 74 ] {} <Fatal> BaseDaemon: 24. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/type_traits:3519: decltype(std::__1::forward<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(fp)()) std::__1::__invoke<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&) @ 0x1cf9f47d in /usr/bin/clickhouse
2020.06.08 16:53:02.812032 [ 75 ] {} <Fatal> BaseDaemon: 29. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:633: DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*) @ 0x1cf9ccf5 in /usr/bin/clickhouse
2020.06.08 16:53:02.835429 [ 74 ] {} <Fatal> BaseDaemon: 25. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/__functional_base:349: void std::__1::__invoke_void_return_wrapper<void>::__call<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&>(DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1&) @ 0x1cf9f42d in /usr/bin/clickhouse
2020.06.08 16:53:02.850103 [ 75 ] {} <Fatal> BaseDaemon: 30. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:546: DB::PipelineExecutor::executeSingleThread(unsigned long, unsigned long) @ 0x1cf9d507 in /usr/bin/clickhouse
2020.06.08 16:53:02.873647 [ 74 ] {} <Fatal> BaseDaemon: 26. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1540: std::__1::__function::__alloc_func<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1, std::__1::allocator<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1>, void ()>::operator()() @ 0x1cf9f3fd in /usr/bin/clickhouse
2020.06.08 16:53:02.888132 [ 75 ] {} <Fatal> BaseDaemon: 31. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:812: DB::PipelineExecutor::executeImpl(unsigned long) @ 0x1cf9ba60 in /usr/bin/clickhouse
2020.06.08 16:53:02.910534 [ 74 ] {} <Fatal> BaseDaemon: 27. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1714: std::__1::__function::__func<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1, std::__1::allocator<DB::PipelineExecutor::addJob(DB::PipelineExecutor::ExecutionState*)::$_1>, void ()>::operator()() @ 0x1cf9e55e in /usr/bin/clickhouse
2020.06.08 16:53:02.910884 [ 74 ] {} <Fatal> BaseDaemon: 28. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:1867: std::__1::__function::__value_func<void ()>::operator()() const @ 0x157c7455 in /usr/bin/clickhouse
2020.06.08 16:53:02.911045 [ 74 ] {} <Fatal> BaseDaemon: 29. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../contrib/libcxx/include/functional:2473: std::__1::function<void ()>::operator()() const @ 0x157c6355 in /usr/bin/clickhouse
2020.06.08 16:53:02.947298 [ 74 ] {} <Fatal> BaseDaemon: 30. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:633: DB::PipelineExecutor::executeStepImpl(unsigned long, unsigned long, std::__1::atomic<bool>*) @ 0x1cf9ccf5 in /usr/bin/clickhouse
2020.06.08 16:53:02.983363 [ 74 ] {} <Fatal> BaseDaemon: 31. /home/mfilimonov/workspace/ClickHouse/build-vscode/clang_8_Debug/../../src/Processors/Executors/PipelineExecutor.cpp:546: DB::PipelineExecutor::executeSingleThread(unsigned long, unsigned long) @ 0x1cf9d507 in /usr/bin/clickhouse

@pytest.mark.timeout(180)
def test_kafka_json_as_string(kafka_cluster):
    kafka_produce('kafka_json_as_string', ['{"t": 123, "e": {"x": "woof"} }', '{"t": 123, "e": {"x": "woof"} }', '{"t": 124, "e": {"x": "test"} }'])

    instance.query('''
        CREATE TABLE test.kafka (field String)
            ENGINE = Kafka
            SETTINGS kafka_broker_list = 'kafka1:19092',
                     kafka_topic_list = 'kafka_json_as_string',
                     kafka_group_name = 'kafka_json_as_string',
                     kafka_format = 'JSONAsString',
                     kafka_flush_interval_ms=1000;
        ''')

    time.sleep(9)

    result = instance.query('SELECT * FROM test.kafka;')
    print(result)

Metadata

Metadata

Assignees

Labels

bugConfirmed user-visible misbehaviour in official releasecomp-formatsInput/output formats (CSV/JSON/Parquet/ORC/Arrow/Protobuf/etc.).comp-message-queuesMessage queue integrations (Kafka, RabbitMQ, NATS table engines for stream ingestion/egress).

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions