Skip to content

TSAN race in librdkafka: rd_avg_calc vs rd_avg_rollover #60939

@al13n321

Description

@al13n321

https://s3.amazonaws.com/clickhouse-test-reports/60878/37d7d80fb8950bb86f6c66ceb2f40e8a889570eb/integration_tests__tsan__[6_6]//home/ubuntu/actions-runner/_work/_temp/test/output_dir/integration_run_parallel4_0.log

E           WARNING: ThreadSanitizer: data race (pid=1)
E             Read of size 4 at 0x7b78000ea568 by thread T982:
E               #0 rd_avg_calc build_docker/./contrib/librdkafka/src/rdavg.h:101:30 (clickhouse+0x1d5dedda) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #1 rd_kafka_broker_timeout_scan build_docker/./contrib/librdkafka/src/rdkafka_broker.c:880:25 (clickhouse+0x1d5dedda)
E               #2 rd_kafka_broker_ops_io_serve build_docker/./contrib/librdkafka/src/rdkafka_broker.c:3416:17 (clickhouse+0x1d5dedda)
E               #3 rd_kafka_broker_consumer_serve build_docker/./contrib/librdkafka/src/rdkafka_broker.c:4975:17 (clickhouse+0x1d5da59e) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #4 rd_kafka_broker_serve build_docker/./contrib/librdkafka/src/rdkafka_broker.c:5080:17 (clickhouse+0x1d5da59e)
E               #5 rd_kafka_broker_thread_main build_docker/./contrib/librdkafka/src/rdkafka_broker.c:5237:25 (clickhouse+0x1d5d3d59) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #6 _thrd_wrapper_function build_docker/./contrib/librdkafka/src/tinycthread.c:576:9 (clickhouse+0x1d6dbcbb) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E           
E             Previous write of size 8 at 0x7b78000ea568 by thread T983 (mutexes: read M0, write M1, write M2):
E               #0 __tsan_memset <null> (clickhouse+0x709cad4) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #1 rd_avg_rollover build_docker/./contrib/librdkafka/src/rdavg.h:170:2 (clickhouse+0x1d5f8c9d) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #2 rd_kafka_stats_emit_avg build_docker/./contrib/librdkafka/src/rdkafka.c:1354:9 (clickhouse+0x1d5f8c9d)
E               #3 rd_kafka_stats_emit_all build_docker/./contrib/librdkafka/src/rdkafka.c:1717:17 (clickhouse+0x1d5f73cb) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #4 rd_kafka_stats_emit_tmr_cb build_docker/./contrib/librdkafka/src/rdkafka.c:1898:2 (clickhouse+0x1d5f73cb)
E               #5 rd_kafka_timers_run build_docker/./contrib/librdkafka/src/rdkafka_timer.c:288:4 (clickhouse+0x1d6c60ca) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #6 rd_kafka_thread_main build_docker/./contrib/librdkafka/src/rdkafka.c:2021:3 (clickhouse+0x1d5f3129) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)
E               #7 _thrd_wrapper_function build_docker/./contrib/librdkafka/src/tinycthread.c:576:9 (clickhouse+0x1d6dbcbb) (BuildId: edd3f4e5f0cd284022b08fc33b68c99dea3d6f01)

test_system_kafka_consumers_rebalance_mv fails around once a day because of it: link

(Not to be confused with another race in librdkafka that was fixed 3 months ago: #56043)

Metadata

Metadata

Assignees

No one assigned

    Labels

    comp-message-queuesMessage queue integrations (Kafka, RabbitMQ, NATS table engines for stream ingestion/egress).duplicate

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions