Project

General

Profile

Actions
Copy link

Bug #64710

closed

kafka: RGW hangs when broker is down for no persistent notifications

Added by Yuval Lifshitz about 2 years ago. Updated 5 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Yuval Lifshitz
Target version:
-
% Done:

0%

Source:
Backport:
squid, reef, quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
55952
Tags (freeform):
Merge Commit:
e08c812a7e37bbf33498d82dadd970f36f1a86f0
Fixed In:
v19.3.0-482-ge08c812a7e
Released In:
v20.2.0~3380
Upkeep Timestamp:
2025-11-01T01:20:01+00:00

Description

based on this comment: https://github.com/ceph/ceph/pull/55051#issuecomment-1961950841
the main issue is that the librdkafka timeout is set by defult to 5 minutes.
due to the 30 seconds idle timeout, we release the pending coroutines after 30 seconds.
but this is still causing the RGW to hang (as it exhaust all of its front-end connections).

Related issues 3 (0 open — 3 closed)

Copied to rgw - Backport #64885: reef: kafka: RGW hangs when broker is down for no persistent notificationsResolvedYuval LifshitzActions
Copied to rgw - Backport #64886: quincy: kafka: RGW hangs when broker is down for no persistent notificationsResolvedYuval LifshitzActions
Copied to rgw - Backport #64887: squid: kafka: RGW hangs when broker is down for no persistent notificationsResolvedYuval LifshitzActions
Actions
Copy link
 #1

Updated by Yuval Lifshitz about 2 years ago

  • Pull request ID set to 55952
Actions
Copy link
 #2

Updated by Yuval Lifshitz about 2 years ago

  • Backport changed from squid, reef, quincy to squid, reef, quincy, pacific
Actions
Copy link
 #3

Updated by Casey Bodley about 2 years ago

  • Status changed from New to Fix Under Review
  • Backport changed from squid, reef, quincy, pacific to squid, reef, quincy
Actions
Copy link
 #4

Updated by Yuval Lifshitz about 2 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #5

Updated by Upkeep Bot about 2 years ago

  • Copied to Backport #64885: reef: kafka: RGW hangs when broker is down for no persistent notifications added
Actions
Copy link
 #6

Updated by Upkeep Bot about 2 years ago

  • Copied to Backport #64886: quincy: kafka: RGW hangs when broker is down for no persistent notifications added
Actions
Copy link
 #7

Updated by Upkeep Bot about 2 years ago

  • Copied to Backport #64887: squid: kafka: RGW hangs when broker is down for no persistent notifications added
Actions
Copy link
 #9

Updated by Yuval Lifshitz about 1 year ago

  • Status changed from Pending Backport to Resolved
Actions
Copy link
 #10

Updated by Upkeep Bot 8 months ago

  • Merge Commit set to e08c812a7e37bbf33498d82dadd970f36f1a86f0
  • Fixed In set to v19.3.0-482-ge08c812a7e3
  • Upkeep Timestamp set to 2025-07-11T15:09:26+00:00
Actions
Copy link
 #11

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-482-ge08c812a7e3 to v19.3.0-482-ge08c812a7e
  • Upkeep Timestamp changed from 2025-07-11T15:09:26+00:00 to 2025-07-14T23:10:31+00:00
Actions
Copy link
 #12

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~3380
  • Upkeep Timestamp changed from 2025-07-14T23:10:31+00:00 to 2025-11-01T01:20:01+00:00
Actions
Copy link

Also available in: Atom PDF