Project

General

Profile

Actions
Copy link

Bug #66017

closed

Notification kafka: Crashes observed when rd_kafka_produce returns any fatal error

Added by Krunal Chheda almost 2 years ago. Updated 5 months ago.

Status:
Resolved
Priority:
High
Assignee:
Yuval Lifshitz
Target version:
-
% Done:

0%

Source:
Community (dev)
Backport:
squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
56033
Tags (freeform):
Merge Commit:
5717482a06d4f2cc26437a324049beff17a92d8c
Fixed In:
v19.3.0-2398-g5717482a06
Released In:
v20.2.0~2827
Upkeep Timestamp:
2025-11-01T01:22:10+00:00

Description

if the rd_kafka_produce produces a fatal error it returns -1 and then code destroys the connection & rd_kafka_topic_t, but does not remove it from vector list and then next code uses the deleted topic from the vector as it was not removed and crashes.
the make `rd_kafka_produce` a fatal error, we are able to do it by just revoking the credential of the kafka user.
Also looking at the rd_kafka_produce documentation, there are multiple use-cases when the rd_kafka_produce can return fatal error (-1) which will cause this crash to happen.

Related issues 2 (0 open — 2 closed)

Related to rgw - Bug #65486: valgrind error on kafka shutdownDuplicateYuval Lifshitz

Actions
Copied to rgw - Backport #66288: squid: Notification kafka: Crashes observed when rd_kafka_produce returns any fatal errorResolvedYuval LifshitzActions
Actions
Copy link
 #1

Updated by Krunal Chheda almost 2 years ago

call stack -->

2024-05-09T19:26:25.043+0000 7fc0852bb700 -1 ** Caught signal (Segmentation fault) *
in thread 7fc0852bb700 thread_name:kafka_manager

1: /lib64/libpthread.so.0(0x12ce0) [0x7fc8d01c3ce0]
 2: /lib64/libc.so.6(+0xccca7) [0x7fc8cf48cca7]
 3: (operator==(rd_kafka_topic_s const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x28) [0x55665aae0238]
 4: (rgw::kafka::Manager::publish_internal(rgw::kafka::message_wrapper_t*)+0xd26) [0x55665aae90f6]
 5: (rgw::kafka::Manager::run()+0x584) [0x55665aae5864]
 6: /lib64/libstdc+.so.6(+0xc2ba3) [0x7fc8cfde2ba3]
 7: /lib64/libpthread.so.0(+0x81cf) [0x7fc8d01b91cf]
 8: clone()
Actions
Copy link
 #2

Updated by Casey Bodley almost 2 years ago

  • Status changed from New to Fix Under Review
Actions
Copy link
 #3

Updated by Yuval Lifshitz almost 2 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #4

Updated by Upkeep Bot almost 2 years ago

  • Copied to Backport #66288: squid: Notification kafka: Crashes observed when rd_kafka_produce returns any fatal error added
Actions
Copy link
 #6

Updated by Casey Bodley almost 2 years ago

  • Related to Bug #65486: valgrind error on kafka shutdown added
Actions
Copy link
 #7

Updated by Upkeep Bot 9 months ago

  • Status changed from Pending Backport to Resolved
  • Upkeep Timestamp set to 2025-07-08T18:35:54+00:00
Actions
Copy link
 #8

Updated by Upkeep Bot 8 months ago

  • Merge Commit set to 5717482a06d4f2cc26437a324049beff17a92d8c
  • Fixed In set to v19.3.0-2398-g5717482a06
  • Upkeep Timestamp changed from 2025-07-08T18:35:54+00:00 to 2025-08-02T04:51:18+00:00
Actions
Copy link
 #9

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~2827
  • Upkeep Timestamp changed from 2025-08-02T04:51:18+00:00 to 2025-11-01T01:22:10+00:00
Actions
Copy link

Also available in: Atom PDF