Project

General

Profile

Actions

Feature #61334

open

cephfs-mirror: use snapdiff api for efficient tree traversal

Added by Venky Shankar almost 3 years ago. Updated 5 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
Performance/Resource Usage
Target version:
% Done:

0%

Source:
Community (dev)
Backport:
reef,squid
Reviewed:
Affected Versions:
Component(FS):
cephfs-mirror, tools
Labels (FS):
Pull request ID:
Tags (freeform):
Fixed In:
v19.3.0-993-ge2d3bf81da
Released In:
v20.2.0~3228
Upkeep Timestamp:
2025-11-01T00:55:38+00:00

Description

With https://github.com/ceph/ceph/pull/43546 merged, cephfs-mirror can make use the snapdiff api (via readdir_snapdiff) to efficiently traverse the directory tree between two snapshots.

This should hugely improve performance when only a handful of files have changed between two consecutive snapshots.


Related issues 3 (1 open2 closed)

Blocked by CephFS - Bug #63099: cephfs-mirror: ceph_getxattr call always return -61 (ENODATA) in PeerReplayer::synchronizeResolvedJos Collin

Actions
Copied to CephFS - Backport #65222: reef: cephfs-mirror: use snapdiff api for efficient tree traversalIn ProgressJos CollinActions
Copied to CephFS - Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversalResolvedJos CollinActions
Actions #1

Updated by Jos Collin almost 3 years ago

  • Assignee set to Jos Collin
Actions #2

Updated by Jos Collin over 2 years ago

  • Description updated (diff)
  • Status changed from New to In Progress
Actions #3

Updated by Venky Shankar over 2 years ago

Jos,

The crux of the changes will be in PeerReplayer::do_synchronize() which if you see does:

sync_stack.emplace(SyncEntry(".", tdirp, tstx)); <<< put "/" on the top of the stack

      while (true) {
        r = ceph_readdirplus_r(m_local_mount, entry.dirp, &de, &stx,
                               CEPH_STATX_MODE | CEPH_STATX_UID | CEPH_STATX_GID |
                               CEPH_STATX_SIZE | CEPH_STATX_ATIME | CEPH_STATX_MTIME,
                               AT_STATX_DONT_SYNC | AT_SYMLINK_NOFOLLOW, NULL);
        if (r < 0) {
          derr << ": failed to local read directory=" << entry.epath << dendl;
          break;
        }
        if (r == 0) {
          break;
        }

        auto d_name = std::string(de.d_name);
        if (d_name != "." && d_name != "..") {
          e_name = d_name;
          break;
        }
      }

which is going to traverse all indoes in the snapshot - note that the mirror daemon does not blindly sync every single inode as it checks the {m,c}time against the "prev"ious snapshot inode. But, the recursive traversal is hitting every single inode and that's what the readdir_diff call would avoid.

Actions #4

Updated by Jos Collin over 2 years ago

  • Blocked by Bug #63099: cephfs-mirror: ceph_getxattr call always return -61 (ENODATA) in PeerReplayer::synchronize added
Actions #5

Updated by Jos Collin over 2 years ago

  • Pull request ID set to 54633
Actions #6

Updated by Jos Collin over 2 years ago

  • Status changed from In Progress to Fix Under Review
  • Source set to Community (dev)
Actions #7

Updated by Venky Shankar almost 2 years ago

  • Status changed from Fix Under Review to Resolved

Not a candidate for backport right now to allow more bake time in main branch.

Actions #8

Updated by Jos Collin almost 2 years ago

Venky Shankar wrote:

Not a candidate for backport right now to allow more bake time in main branch.

What about this BZ 7.1z1 then: https://bugzilla.redhat.com/show_bug.cgi?id=2214930?

Actions #9

Updated by Venky Shankar almost 2 years ago

Jos Collin wrote:

Venky Shankar wrote:

Not a candidate for backport right now to allow more bake time in main branch.

What about this BZ 7.1z1 then: https://bugzilla.redhat.com/show_bug.cgi?id=2214930?

I'm aware of that. We need to clear some other test failures in fs:mirror that we are seeing.

Actions #10

Updated by Venky Shankar almost 2 years ago

  • Status changed from Resolved to Pending Backport
  • Target version changed from v19.0.0 to v20.0.0
  • Backport set to reef,squid

Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?

Actions #11

Updated by Upkeep Bot almost 2 years ago

  • Copied to Backport #65222: reef: cephfs-mirror: use snapdiff api for efficient tree traversal added
Actions #12

Updated by Upkeep Bot almost 2 years ago

  • Copied to Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversal added
Actions #14

Updated by Jos Collin almost 2 years ago

Venky Shankar wrote:

Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?

ok

Actions #15

Updated by Upkeep Bot 9 months ago

  • Merge Commit set to e2d3bf81da0f7f62956a2a34121ec4a39252ff88
  • Fixed In set to v19.3.0-993-ge2d3bf81da0
  • Upkeep Timestamp set to 2025-07-09T16:44:16+00:00
Actions #16

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v19.3.0-993-ge2d3bf81da0 to v19.3.0-993-ge2d3bf81da
  • Upkeep Timestamp changed from 2025-07-09T16:44:16+00:00 to 2025-07-14T17:41:58+00:00
Actions #17

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~3228
  • Upkeep Timestamp changed from 2025-07-14T17:41:58+00:00 to 2025-11-01T00:55:38+00:00
Actions

Also available in: Atom PDF