Project

General

Profile

Actions
Copy link

Bug #71261

open

osdc: idle librados client can lead to Objecter getting way behind in osdmaps

Added by Casey Bodley 11 months ago. Updated 8 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Nitzan Mordechai
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
squid tentacle
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Objecter
Pull request ID:
63425
Tags (freeform):
backport_processed
Merge Commit:
47c922c2b71fd88b7150ff421c70e2e7fa770b6b
Fixed In:
v20.3.0-1119-g47c922c2b7
Released In:
Upkeep Timestamp:
2025-07-14T20:45:09+00:00

Description

as observed in https://tracker.ceph.com/issues/71239, with objecter debug logs summarized in https://gist.github.com/cbodley/aeb273e19ab26dda20f1072126c890a2

by the time a librados request is submitted, it was on osdmap epoch 109 of 603, and took over 45 seconds fetching and applying incremental osdmaps before it was able to successfully submit the request and get a response

this delay caused timeouts in the 'ceph api tests'

Related issues 4 (2 open — 2 closed)

Related to rgw - Bug #71239: list_realms regression after change to ConfigStore ResolvedCasey Bodley

Actions
Related to RADOS - Bug #71931: rados/test_health_warnings.sh: OSD got stuck down and never came back upPending BackportNaveen Naidu

Actions
Copied to RADOS - Backport #71959: tentacle: osdc: idle librados client can lead to Objecter getting way behind in osdmapsResolvedNitzan MordechaiActions
Copied to RADOS - Backport #71960: squid: osdc: idle librados client can lead to Objecter getting way behind in osdmapsIn ProgressNitzan MordechaiActions
Actions
Copy link
 #1

Updated by Casey Bodley 11 months ago

  • Related to Bug #71239: list_realms regression after change to ConfigStore added
Actions
Copy link
 #2

Updated by Radoslaw Zarzynski 10 months ago

  • Assignee set to Nitzan Mordechai

Hi Nitzan! Would you mind taking a look?

Actions
Copy link
 #3

Updated by Nitzan Mordechai 10 months ago

  • Status changed from New to In Progress
Actions
Copy link
 #4

Updated by Nitzan Mordechai 10 months ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 63425
Actions
Copy link
 #5

Updated by Radoslaw Zarzynski 10 months ago

Reapproved, already sent to QA.

Actions
Copy link
 #6

Updated by Laura Flores 9 months ago

Fix ready for QA

Actions
Copy link
 #7

Updated by Casey Bodley 9 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Backport set to squid tentacle
Actions
Copy link
 #8

Updated by Upkeep Bot 9 months ago

  • Copied to Backport #71959: tentacle: osdc: idle librados client can lead to Objecter getting way behind in osdmaps added
Actions
Copy link
 #9

Updated by Upkeep Bot 9 months ago

  • Copied to Backport #71960: squid: osdc: idle librados client can lead to Objecter getting way behind in osdmaps added
Actions
Copy link
 #10

Updated by Upkeep Bot 9 months ago

  • Tags (freeform) set to backport_processed
Actions
Copy link
 #11

Updated by Upkeep Bot 9 months ago

  • Merge Commit set to 47c922c2b71fd88b7150ff421c70e2e7fa770b6b
  • Fixed In set to v20.3.0-1119-g47c922c2b71
  • Upkeep Timestamp set to 2025-07-08T14:46:13+00:00
Actions
Copy link
 #12

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v20.3.0-1119-g47c922c2b71 to v20.3.0-1119-g47c922c2b71f
  • Upkeep Timestamp changed from 2025-07-08T14:46:13+00:00 to 2025-07-14T15:20:33+00:00
Actions
Copy link
 #13

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v20.3.0-1119-g47c922c2b71f to v20.3.0-1119-g47c922c2b7
  • Upkeep Timestamp changed from 2025-07-14T15:20:33+00:00 to 2025-07-14T20:45:09+00:00
Actions
Copy link
 #14

Updated by Yaarit Hatuka 4 months ago

  • Related to Bug #71931: rados/test_health_warnings.sh: OSD got stuck down and never came back up added
Actions
Copy link

Also available in: Atom PDF