Project

General

Profile

Actions

Bug #69345

open

For an irreparable hard link scrub is not able to identify the damage

Added by Md Mahamudur Rahaman Sajib about 1 year ago. Updated 6 months ago.

Status:
Pending Backport
Priority:
Normal
Category:
Administration/Usability
Target version:
% Done:

0%

Source:
Community (dev)
Backport:
tentacle,squid,reef
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
MDS
Labels (FS):
scrub
Pull request ID:
Tags (freeform):
backport_processed
Fixed In:
v20.3.0-3322-g7ce4252964
Released In:
Upkeep Timestamp:
2025-09-29T14:10:50+00:00

Description

We have a known damage in a production cluster, https://tracker.ceph.com/issues/60986#note-15

We know that damage is irreparable as there has been a data loss.

I am able to reproduce it easily by some hack.

  1. Create a directory e.g, /dir
  2. Create a file inside /dir , e.g. /dir/x.txt
  3. Create a link e.g. /y.txt of /dir/x.txt
  4. remove the associdated inode object of /dir from the meta pool(Just to create the damage forcefully)
  5. Try to stat the /y.txt and damage will occur.
  6. Remove the damage from the damage list
  7. Scrub the root directory, damage list will remain empty.

I found in the ScrubStack.cc code,

      if (dnl->is_primary()) {
    _enqueue(dnl->get_inode(), header, false);
      } else if (dnl->is_remote()) {
    // TODO: check remote linkage
      }

If dnl->is_remote() it is doing nothing, there is TODO marked. What should be ideal in this case as doing scrub will make redundant scrub operations on a single inode. In that case can we just try to open the inode using MDCache::open_remote_dentry(CDentry *dn, bool projected, MDSContext *fin, bool want_xlocked)(This is they way used for finding damage while path traversal)?

Related issues 3 (3 open0 closed)

Copied to CephFS - Backport #73303: squid: For an irreparable hard link scrub is not able to identify the damageQA TestingMd Mahamudur Rahaman SajibActions
Copied to CephFS - Backport #73304: tentacle: For an irreparable hard link scrub is not able to identify the damageQA TestingMd Mahamudur Rahaman SajibActions
Copied to CephFS - Backport #73373: reef: For an irreparable hard link scrub is not able to identify the damageIn ProgressMd Mahamudur Rahaman SajibActions
Actions #1

Updated by Md Mahamudur Rahaman Sajib about 1 year ago

  • Subject changed from For a irreparable hard link scrub is not able to identify the damage to For an irreparable hard link scrub is not able to identify the damage
Actions #2

Updated by Md Mahamudur Rahaman Sajib about 1 year ago ยท Edited

Scrub the root directory, damage list will remain empty.

Whenever we tried to stat /y.txt damage will re-appear again.

Actions #3

Updated by Md Mahamudur Rahaman Sajib about 1 year ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 61173
Actions #4

Updated by Venky Shankar about 1 year ago

  • Category set to Administration/Usability
  • Target version set to v20.0.0
  • Source set to Community (dev)
  • Backport changed from reef, quincy, squid to reef,squid
Actions #5

Updated by Venky Shankar 7 months ago

  • Target version changed from v20.0.0 to v21.0.0
  • Backport changed from reef,squid to tentacle,squid,reedf
Actions #6

Updated by Venky Shankar 6 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Backport changed from tentacle,squid,reedf to tentacle,squid
Actions #7

Updated by Upkeep Bot 6 months ago

  • Merge Commit set to 7ce425296461505a04bb69939b366436230b63c0
  • Fixed In set to v20.3.0-3322-g7ce4252964
  • Upkeep Timestamp set to 2025-09-29T14:10:50+00:00
Actions #8

Updated by Upkeep Bot 6 months ago

  • Copied to Backport #73303: squid: For an irreparable hard link scrub is not able to identify the damage added
Actions #9

Updated by Upkeep Bot 6 months ago

  • Copied to Backport #73304: tentacle: For an irreparable hard link scrub is not able to identify the damage added
Actions #10

Updated by Upkeep Bot 6 months ago

  • Tags (freeform) set to backport_processed
Actions #11

Updated by Md Mahamudur Rahaman Sajib 6 months ago

  • Backport changed from tentacle,squid to tentacle,squid,reef
Actions #12

Updated by Md Mahamudur Rahaman Sajib 6 months ago

  • Copied to Backport #73373: reef: For an irreparable hard link scrub is not able to identify the damage added
Actions

Also available in: Atom PDF