Skip to content

tools/cephfs-mirror: eliminate redundant ceph_close() call#60667

Merged
vshankar merged 2 commits intoceph:mainfrom
ifed01:wip-ifed-fix-cephfs-mirror
Dec 16, 2024
Merged

tools/cephfs-mirror: eliminate redundant ceph_close() call#60667
vshankar merged 2 commits intoceph:mainfrom
ifed01:wip-ifed-fix-cephfs-mirror

Conversation

@ifed01
Copy link
Contributor

@ifed01 ifed01 commented Nov 7, 2024

Fixes: https://tracker.ceph.com/issues/68853

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@ifed01 ifed01 requested a review from vshankar November 7, 2024 17:34
@github-actions github-actions bot added the cephfs Ceph File System label Nov 7, 2024
@ifed01 ifed01 requested a review from joscollin November 7, 2024 17:34
@ifed01 ifed01 added the bug-fix label Nov 7, 2024
@vshankar
Copy link
Contributor

jenkins test api

@vshankar
Copy link
Contributor

@joscollin PTAL for review and approve if the change looks good.

@joscollin
Copy link
Member

@vshankar Will check soon.

@ifed01
Copy link
Contributor Author

ifed01 commented Nov 13, 2024

jenkins test api

@vshankar
Copy link
Contributor

@vshankar Will check soon.

Anything on this @joscollin ?

@joscollin
Copy link
Member

@vshankar Will check soon.

Anything on this @joscollin ?

Currently reviewing it. The new changes looks good at the first look. Checking in detail.

@ifed01 ifed01 force-pushed the wip-ifed-fix-cephfs-mirror branch from 5dc24fa to b9d40b1 Compare November 22, 2024 16:38
@sajibreadd
Copy link
Member

sajibreadd commented Nov 25, 2024

ceph_closedir the fd internally then why are we closing the fd in opendirat?

Copy link
Member

@joscollin joscollin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, looks good to me.

@ifed01 ifed01 force-pushed the wip-ifed-fix-cephfs-mirror branch from b9d40b1 to 8ef5012 Compare December 2, 2024 17:22
Copy link
Member

@joscollin joscollin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, dout is only for success case. But not a big deal. LGTM.

Copy link
Member

@joscollin joscollin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ifed01 Oops, now there's a redundant commit a0df949. Could you please fix this?

@ifed01 ifed01 force-pushed the wip-ifed-fix-cephfs-mirror branch from 8ef5012 to 4234eb7 Compare December 3, 2024 12:35
Likely ceph#55619 missed that.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
ceph#55619 eliminates the need for ceph_close() call after successful ceph_fdopendir() one.
And introduces automatic file descriptor's close when corresponding ceph_closedir() is called.
That hasty ceph_close() call makes file descriptor
available for a new allocation which might conflict with the automatic fd close in the above ceph_closedir().

Full PeerReplayer::do_synchronize() has been reworked to close fds
properly, depending on whether ceph_fdopendir() has been already applied
to them.

Additionally for the sake of uniformity this reworks incremental do_synchronize()
in a way to do final fd closings similar to full implementation.

Plus this effectively reverts
ceph@bd78bdc
as it looks like a wrong approach to fight broken file descriptor
references. No much sense in reopening of the current snapshot's root folder
on each new entry processing. Instead this patch just doesn't close it from the
beginning.

Fixes: https://tracker.ceph.com/issues/68853

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
@ifed01 ifed01 force-pushed the wip-ifed-fix-cephfs-mirror branch from 4234eb7 to 8c5999f Compare December 3, 2024 12:42
@ifed01
Copy link
Contributor Author

ifed01 commented Dec 3, 2024

@ifed01 Oops, now there's a redundant commit a0df949. Could you please fix this?

@joscollin - yeah, squashed commits around ceph_close() into a single one.
Added another trivial one to get rid off build warning in Client.cc. It's irrelevant to cephfs-mirror just want to piggy-back this PR to cut the path if you don't mind...

@joscollin
Copy link
Member

@ifed01 Oops, now there's a redundant commit a0df949. Could you please fix this?

@joscollin - yeah, squashed commits around ceph_close() into a single one.

Thanks.

Added another trivial one to get rid off build warning in Client.cc. It's irrelevant to cephfs-mirror just want to piggy-back this PR to cut the path if you don't mind...

We won't catch that in the fs:mirror tests, but it certainly needs QA. So could you please create a new PR for that fix?

@ifed01 ifed01 force-pushed the wip-ifed-fix-cephfs-mirror branch from 8c5999f to 7a747bc Compare December 3, 2024 13:04
@ifed01
Copy link
Contributor Author

ifed01 commented Dec 3, 2024

Added another trivial one to get rid off build warning in Client.cc. It's irrelevant to cephfs-mirror just want to piggy-back this PR to cut the path if you don't mind...

We won't catch that in the fs:mirror tests, but it certainly needs QA. So could you please create a new PR for that fix?

Done

Copy link
Member

@joscollin joscollin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@rishabh-d-dave Could you please add this PR to your next QA batch run? Thx.

@rishabh-d-dave rishabh-d-dave added the wip-rishabh-testing Rishabh's testing label label Dec 3, 2024
@ifed01
Copy link
Contributor Author

ifed01 commented Dec 3, 2024

jenkins test api

@joscollin
Copy link
Member

@rishabh-d-dave This is a known issue [1] https://tracker.ceph.com/issues/68567.

@joscollin
Copy link
Member

@rishabh-d-dave
The QA has a failure:
https://pulpito.ceph.com/rishabh-2024-12-09_14:46:35-fs-wip-rishabh-testing-20241204.130505-testing-default-smithi/8026609/

This is a known issue, which is tracked here: https://tracker.ceph.com/issues/68567. Could you please drop that failed test and QA again, so that we could get this PR in?

@rishabh-d-dave
Copy link
Contributor

@rishabh-d-dave The QA has a failure: https://pulpito.ceph.com/rishabh-2024-12-09_14:46:35-fs-wip-rishabh-testing-20241204.130505-testing-default-smithi/8026609/

This is a known issue, which is tracked here: https://tracker.ceph.com/issues/68567. Could you please drop that failed test

Drop that test how?

and QA again, so that we could get this PR in?

QA again why? We already did a full QA run, there are no other mirror related failures there.

joscollin pushed a commit to joscollin/ceph that referenced this pull request Dec 12, 2024
* refs/pull/60667/head:
	cephfs-mirror: remove redundant ceph_close() calls.
	cephfs/client: dir_reset_t::reset() - add missing fd reset.

Reviewed-by: Jos Collin <jcollin@redhat.com>
@joscollin
Copy link
Member

@rishabh-d-dave The QA has a failure: https://pulpito.ceph.com/rishabh-2024-12-09_14:46:35-fs-wip-rishabh-testing-20241204.130505-testing-default-smithi/8026609/
This is a known issue, which is tracked here: https://tracker.ceph.com/issues/68567. Could you please drop that failed test

Drop that test how?

and QA again, so that we could get this PR in?

QA again why? We already did a full QA run, there are no other mirror related failures there.

#61068
That's the fix for the failure. I'll take up the QA for this PR, as I have more fixes coming in.

@joscollin
Copy link
Member

@vshankar
QA passed: https://pulpito.ceph.com/jcollin-2024-12-13_12:51:29-fs:mirror-wip-jcollin-testing-20241213.065332-main-distro-default-smithi/.

Please merge this PR.

@vshankar vshankar merged commit 23d38a1 into ceph:main Dec 16, 2024
@joscollin joscollin removed their assignment Dec 16, 2024
@ifed01 ifed01 deleted the wip-ifed-fix-cephfs-mirror branch December 16, 2024 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix cephfs Ceph File System wip-rishabh-testing Rishabh's testing label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants