Project

General

Profile

Actions

Bug #67601

open

qa: cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline

Added by Milind Changire over 1 year ago. Updated about 2 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
Correctness/Safety
Target version:
% Done:

0%

Source:
Q/A
Backport:
squid,reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Tags (freeform):
temp-assign backport_processed
Fixed In:
v19.3.0-4617-g1650722139
Released In:
v20.2.0~2125
Upkeep Timestamp:
2025-11-01T01:02:39+00:00


Related issues 2 (1 open1 closed)

Copied to CephFS - Backport #67836: squid: qa: cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offlineQA TestingPatrick DonnellyActions
Copied to CephFS - Backport #67837: reef: qa: cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offlineResolvedPatrick DonnellyActions
Actions #1

Updated by Venky Shankar over 1 year ago

  • Category set to Correctness/Safety
  • Status changed from New to Need More Info
  • Assignee set to Milind Changire

Milind, please check this too if its related to the PR under test (uninline data) on scrub.

Actions #2

Updated by Patrick Donnelly over 1 year ago

  • Status changed from Need More Info to In Progress
  • Assignee changed from Milind Changire to Patrick Donnelly
Actions #3

Updated by Patrick Donnelly over 1 year ago

  • Status changed from In Progress to Fix Under Review
  • Target version set to v20.0.0
  • Source set to Q/A
  • Backport set to squid,reef
  • Pull request ID set to 59309
Actions #4

Updated by Venky Shankar over 1 year ago

  • Status changed from Fix Under Review to Pending Backport
Actions #5

Updated by Upkeep Bot over 1 year ago

  • Copied to Backport #67836: squid: qa: cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline added
Actions #6

Updated by Upkeep Bot over 1 year ago

  • Copied to Backport #67837: reef: qa: cluster [ERR] Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline added
Actions #7

Updated by Upkeep Bot over 1 year ago

  • Tags (freeform) set to backport_processed
Actions #9

Updated by Milind Changire about 1 year ago ยท Edited

squid:
https://pulpito.ceph.com/vshankar-2024-11-23_10:32:29-fs-wip-vshankar-testing-20241122.180955-squid-debug-testing-default-smithi/8005737

This is mostly due to a preceeding "fs rm" command:

2024-11-23T18:19:59.332+0000 7f20587c4640  7 mon.a@0(leader).mds e31 prepare_update mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0)
2024-11-23T18:19:59.332+0000 7f20587c4640  7 mon.a@0(leader).mds e31 print_map
e32
btime 2024-11-23T18:19:58:269360+0000
enable_multiple, ever_enabled_multiple: 1,1
default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes}
legacy client fscid: -1

No filesystems configured
Standby daemons:

[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.12:6834/3927056531,v1:172.21.15.12:6835/3927056531] compat {c=[1],r=[1],i=[1fff]}]
[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.12:6836/2460436318,v1:172.21.15.12:6837/2460436318] compat {c=[1],r=[1],i=[1fff]}]
[mds.c{-1:19620} state up:standby seq 1 addr [v2:172.21.15.113:6833/227986182,v1:172.21.15.113:6835/227986182] compat {c=[1],r=[1],i=[1fff]}]
[mds.h{-1:19644} state up:standby seq 1 addr [v2:172.21.15.70:6836/1936421792,v1:172.21.15.70:6837/1936421792] compat {c=[1],r=[1],i=[1fff]}]
[mds.f{-1:28723} state up:standby seq 1 addr [v2:172.21.15.113:6832/2736917072,v1:172.21.15.113:6834/2736917072] compat {c=[1],r=[1],i=[1fff]}]
[mds.g{-1:28732} state up:standby seq 1 addr [v2:172.21.15.12:6840/3275729468,v1:172.21.15.12:6841/3275729468] compat {c=[1],r=[1],i=[1fff]}]
[mds.i{-1:28753} state up:standby seq 1 addr [v2:172.21.15.113:6836/1017697073,v1:172.21.15.113:6838/1017697073] compat {c=[1],r=[1],i=[1fff]}]
[mds.l{-1:28756} state up:standby seq 1 addr [v2:172.21.15.113:6837/958413334,v1:172.21.15.113:6839/958413334] compat {c=[1],r=[1],i=[1fff]}]
[mds.k{-1:28766} state up:standby seq 1 addr [v2:172.21.15.70:6832/3700829433,v1:172.21.15.70:6833/3700829433] compat {c=[1],r=[1],i=[1fff]}]
[mds.a{-1:28778} state up:standby seq 1 addr [v2:172.21.15.12:6838/358449145,v1:172.21.15.12:6839/358449145] compat {c=[1],r=[1],i=[1fff]}]
[mds.e{-1:28796} state up:standby seq 1 addr [v2:172.21.15.70:6834/3768174742,v1:172.21.15.70:6835/3768174742] compat {c=[1],r=[1],i=[1fff]}]
[mds.b{-1:28811} state up:standby seq 1 addr [v2:172.21.15.70:6838/3183466107,v1:172.21.15.70:6839/3183466107] compat {c=[1],r=[1],i=[1fff]}]

squid:
https://pulpito.ceph.com/vshankar-2024-11-23_10:32:29-fs-wip-vshankar-testing-20241122.180955-squid-debug-testing-default-smithi/8005762

Actions #10

Updated by Upkeep Bot 8 months ago

  • Merge Commit set to 1650722139d0ddc26f780e2ab09675290ace8e32
  • Fixed In set to v19.3.0-4617-g1650722139d
  • Upkeep Timestamp set to 2025-07-08T18:45:33+00:00
Actions #11

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-4617-g1650722139d to v19.3.0-4617-g1650722139d0
  • Upkeep Timestamp changed from 2025-07-08T18:45:33+00:00 to 2025-07-14T15:45:52+00:00
Actions #12

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-4617-g1650722139d0 to v19.3.0-4617-g1650722139
  • Upkeep Timestamp changed from 2025-07-14T15:45:52+00:00 to 2025-07-14T21:10:07+00:00
Actions #14

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~2125
  • Upkeep Timestamp changed from 2025-07-14T21:10:07+00:00 to 2025-11-01T01:02:39+00:00
Actions #15

Updated by Venky Shankar about 2 months ago

  • Assignee changed from Patrick Donnelly to Mahesh Mohan
  • Tags (freeform) changed from backport_processed to temp-assign

@Milind Changire is moving away from CephFS development. Assigning this to @Mahesh Mohan in the interim.

Actions #16

Updated by Upkeep Bot about 2 months ago

  • Tags (freeform) changed from temp-assign to temp-assign backport_processed
Actions

Also available in: Atom PDF