Project

General

Profile

Actions
Copy link

Bug #65422

closed

upgrade/quincy-x: "1 pg degraded (PG_DEGRADED)" in cluster log

Added by Laura Flores almost 2 years ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Nitzan Mordechai
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
squid
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
58277
Tags (freeform):
backport_processed
Merge Commit:
1161a9fb43a031cf6230e9f1c52babd996645bf3
Fixed In:
v19.3.0-3645-g1161a9fb43
Released In:
v20.2.0~2424
Upkeep Timestamp:
2025-11-01T01:33:51+00:00
Tags:
cluster-log-warning

Description

/a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648908

[WRN] Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED)" in cluster log

Related issues 3 (1 open — 2 closed)

Related to RADOS - Bug #65521: Add expected warnings in cluster log to ignorelistsClosedLaura Flores

Actions
Related to RADOS - Bug #69135: upgrade/reef-x: "1 osds down (OSD_DOWN)" in cluster logPending BackportIlya Dryomov

Actions
Copied to RADOS - Backport #67138: squid: upgrade/quincy-x: "1 pg degraded (PG_DEGRADED)" in cluster logResolvedNitzan MordechaiActions
Actions
Copy link
 #1

Updated by Venky Shankar almost 2 years ago

  • Project changed from CephFS to RADOS
  • Assignee set to Laura Flores

Laura, handing this back to you since this isn't really cephfs related.

Actions
Copy link
 #2

Updated by Laura Flores almost 2 years ago

Thanks Venky, it was a mistake that I added it there in the first place.

Actions
Copy link
 #3

Updated by Laura Flores almost 2 years ago

Needs to be whitelisted; will bring this issue and others like it to the next RADOS meeting so we can divide up that work.

Actions
Copy link
 #4

Updated by Laura Flores almost 2 years ago 

/a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652502$ tail teuthology.log 
duration: 2631.234354734421
failure_reason: '"2024-04-12T02:00:00.000074+0000 mon.a (mon.0) 1431 : cluster [WRN]
  Health detail: HEALTH_WARN 1 osds down; Degraded data redundancy: 139/1684 objects
  degraded (8.254%), 1 pg degraded" in cluster log'
flavor: default
owner: scheduled_yuriw@teuthology
success: false

2024-04-12T02:18:13.093 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-04-12T02:18:13.184 INFO:teuthology.run:FAIL
Actions
Copy link
 #5

Updated by Laura Flores almost 2 years ago

  • Related to Bug #65521: Add expected warnings in cluster log to ignorelists added
Actions
Copy link
 #6

Updated by Nitzan Mordechai over 1 year ago · Edited

  • Assignee changed from Laura Flores to Nitzan Mordechai
Actions
Copy link
 #7

Updated by Nitzan Mordechai over 1 year ago · Edited

list to ignore will be:

log-ignorelist:
      - POOL_APP_NOT_ENABLED
      - mons down
      - mon down
      - MON_DOWN
      - out of quorum
      - PG_DEGRADED
      - Reduced data availability
      - Degraded data redundancy
Actions
Copy link
 #8

Updated by Nitzan Mordechai over 1 year ago

  • Pull request ID set to 58277
Actions
Copy link
 #9

Updated by Nitzan Mordechai over 1 year ago

  • Backport set to squid
Actions
Copy link
 #10

Updated by Nitzan Mordechai over 1 year ago

/a/yuriw-2024-06-21_13:33:28-upgrade:quincy-x-wip-yuri6-testing-2024-06-20-1620-squid-distro-default-smithi/7766643

Actions
Copy link
 #11

Updated by Neha Ojha over 1 year ago

  • Status changed from New to Fix Under Review
Actions
Copy link
 #12

Updated by Laura Flores over 1 year ago

  • Subject changed from upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log to upgrade/quincy-x: "1 pg degraded (PG_DEGRADED)" in cluster log

Thanks Nitzan! Since the fix also handles stress-split, I'm going to modify the title slightly.

Actions
Copy link
 #13

Updated by Laura Flores over 1 year ago

/a/yuriw-2024-06-28_19:31:25-upgrade:quincy-x-squid-release-distro-default-smithi/7778104

For our records

Actions
Copy link
 #14

Updated by Laura Flores over 1 year ago

  • Copied to Bug #66810: upgrade/reef-x: "1 pg degraded (PG_DEGRADED)" in cluster log added
Actions
Copy link
 #15

Updated by Laura Flores over 1 year ago

  • Copied to deleted (Bug #66810: upgrade/reef-x: "1 pg degraded (PG_DEGRADED)" in cluster log)
Actions
Copy link
 #16

Updated by Radoslaw Zarzynski over 1 year ago

scrub note: already approved, in the QA.

Actions
Copy link
 #17

Updated by Radoslaw Zarzynski over 1 year ago

scrub note: awaits QA.

Actions
Copy link
 #18

Updated by Laura Flores over 1 year ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #19

Updated by Upkeep Bot over 1 year ago

  • Copied to Backport #67138: squid: upgrade/quincy-x: "1 pg degraded (PG_DEGRADED)" in cluster log added
Actions
Copy link
 #20

Updated by Upkeep Bot over 1 year ago

  • Tags (freeform) set to backport_processed
Actions
Copy link
 #21

Updated by Laura Flores over 1 year ago

/a/yuriw-2024-08-15_14:10:18-upgrade:quincy-x-squid-release-distro-default-smithi/7856786

Actions
Copy link
 #22

Updated by Laura Flores over 1 year ago

/a/yuriw-2024-08-29_20:06:20-upgrade:quincy-x-squid-release-distro-default-smithi/7880568

Actions
Copy link
 #23

Updated by Ilya Dryomov over 1 year ago

  • Related to Bug #69135: upgrade/reef-x: "1 osds down (OSD_DOWN)" in cluster log added
Actions
Copy link
 #24

Updated by Upkeep Bot about 1 year ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions
Copy link
 #25

Updated by Laura Flores about 1 year ago

  • Tags set to cluster-log-warning
Actions
Copy link
 #26

Updated by Upkeep Bot 8 months ago

  • Merge Commit set to 1161a9fb43a031cf6230e9f1c52babd996645bf3
  • Fixed In set to v19.3.0-3645-g1161a9fb43a
  • Upkeep Timestamp set to 2025-07-11T13:45:17+00:00
Actions
Copy link
 #27

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-3645-g1161a9fb43a to v19.3.0-3645-g1161a9fb43
  • Upkeep Timestamp changed from 2025-07-11T13:45:17+00:00 to 2025-07-14T23:09:40+00:00
Actions
Copy link
 #28

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~2424
  • Upkeep Timestamp changed from 2025-07-14T23:09:40+00:00 to 2025-11-01T01:33:51+00:00
Actions
Copy link

Also available in: Atom PDF