libcephfs: Nonblocking io by ffilz · Pull Request #48038 · ceph/ceph

ffilz · 2022-09-09T23:03:21Z

This replaces #44991 (async I/O).

This tests as working with Ganesha. I renamed to nonblocking I/O because there was some confusion with Read_Sync_Async that looks a lot better as Read_Sync_Nonblocking. A unit test is included.

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

github-actions · 2022-09-23T07:19:18Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

ljflores · 2022-11-21T17:41:58Z

@ffilz looks like this needs a rebase

ffilz · 2022-12-06T22:03:37Z

@ffilz looks like this needs a rebase

Rebase pushed.

src/osdc/ObjectCacher.h

src/include/buffer.h

src/client/Client.cc

src/client/Client.h

vshankar · 2023-03-15T09:04:23Z

@ffilz We will target this post reef branching and give it enough time to bake in main branch.

github-actions · 2023-04-13T10:14:51Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

mattbenjamin · 2023-05-01T16:06:42Z

@vshankar I'm not certain, but it looks as if this PR hasn't gotten a single review comment from the cephfs team, in 6 months. We are depending on starting a process that allows ganesha devs to get ongoing attention from cephfs devs. Please try to get a review in?

mattbenjamin · 2023-05-01T16:07:55Z

@ffilz this should probably get a rebase. More importantly, there are commits which lack your signed-off-by. Please address that, and try to address the make check failure, after rebasing.

ffilz · 2023-05-03T17:34:19Z

@ffilz this should probably get a rebase. More importantly, there are commits which lack your signed-off-by. Please address that, and try to address the make check failure, after rebasing.

I did a rebase last week. The unsigned commit is something that hopefully isn't actually needed but I needed it to compile. It's got nothing to do with my code... I will remove that commit before expecting the pull request to be merged.

ffilz · 2023-05-03T17:35:43Z

Clicking on the details for the make check failures doesn't lead to any insight on them...

ffilz · 2023-07-14T17:50:26Z

OK, done that. Sorry.

Did you forget pushing?

No, I was in a hurry and didn't pay attention to the error message. It didn't go because the network connection for my VM was bunged up do to the fuse covering my office tripping (and thus killing my switch)...

Should be pushed now...

vshankar · 2023-07-18T10:54:53Z

jenkins retest this please

vshankar · 2023-07-18T14:44:26Z

jenkins test make check

vshankar · 2023-07-18T14:44:37Z

The windows failure is a known issue right now.

vshankar

https://tracker.ceph.com/projects/cephfs/wiki/Main#18-July-2023

vshankar · 2023-07-19T03:58:53Z

jenkins test make check

vshankar · 2023-07-21T05:17:46Z

jenkins test make check

vshankar · 2023-07-21T05:18:02Z

jenkins test windows

vshankar · 2023-07-21T07:07:51Z

Windows jenkins failure fix: #52427

vshankar · 2023-07-24T05:36:00Z

jenkins test windows

vshankar · 2023-07-24T10:09:32Z

@ffilz please rebase and push - the windows failure has been fixed and merged.

For non-blocking I/O, we will want to be able to override block_writes_upfront so rename the member cfg_block_writes_upfront and add an option to pass block_writes_upfront as a parameter along with a member access method so caller can pass cfg_block_writes_upfront. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

These bits of code need to be invoked from a separate spot when we introduce non-blocking I/O, so break them out now. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Change waitfor_caps, waitfor_safe and waitfor_commit to Context list. To make a non-blocking version of fsync (to be used for non-blocking write and commit), we need to be able to signal an arbitrary Context on completion of either of these lists. add_nonblocking_onfinish_to_context_list Adds such a Context to the list. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

The non-blocking flush requires us to be able to re-add to wait_for_caps but if we simply add to the list, we get stuck in an infinite loop. Add a wait_for_caps_pending list to add to, and then when done signalling, we move the wait_for_caps_pending items onto the wait_for_caps list. Also in handle_cap_flush_ack(), we need to complete the caps flushing before signalling since with non-blocking flush, we will be actually examining the caps from the completion rather than signalling a condition variable in the completion. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

We will need the ability to do an non-blocking write that finishes with fsync so we need non-blocking fsync. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

github-actions bot added build/ops cephfs Ceph File System tests labels Sep 9, 2022

ffilz mentioned this pull request Sep 9, 2022

WIP - libcephfs: add async I/O capability #44991

Closed

14 tasks

ffilz requested review from adamemerson, ajarr, batrick, dang, jtlayton, kotreshhr, lxbsz and mattbenjamin September 15, 2022 20:19

github-actions bot added the needs-rebase label Sep 23, 2022

cbodley changed the title ~~Nonblocking io~~ libcephfs: Nonblocking io Oct 3, 2022

cbodley added the feature label Oct 3, 2022

ffilz force-pushed the nonblocking-io branch from dd7a549 to 71632b8 Compare December 6, 2022 22:02

github-actions bot removed the needs-rebase label Dec 6, 2022

adamemerson reviewed Jan 19, 2023

View reviewed changes

github-actions bot added the needs-rebase label Apr 13, 2023

ffilz force-pushed the nonblocking-io branch from efaf69a to 3b4a61f Compare April 26, 2023 20:49

github-actions bot removed the needs-rebase label Apr 26, 2023

vshankar added the ready-to-merge label Jul 17, 2023

vshankar approved these changes Jul 18, 2023

View reviewed changes

vshankar removed the wip-vshankar-testing2 label Jul 18, 2023

ffilz added 13 commits July 24, 2023 11:49

Buffers: Add function to buffer.h to copy bufferlist to an iovec

d3c3bf5

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Client: Break some code into new methods in prep for non-blocking I/O

162935c

These bits of code need to be invoked from a separate spot when we introduce non-blocking I/O, so break them out now. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Client: Add non-blocking helper classes

1210ddf

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Client: Add ll_preadv_pwritev to expose non-blocking I/O to libcephfs

94b55cb

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

libcephfs: Add nonblocking readv/writev I/O interface

b4e39f3

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

test: Add nonblocking I/O client test

6b50da6

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Client: Add non-blocking fsync

a181c1a

We will need the ability to do an non-blocking write that finishes with fsync so we need non-blocking fsync. Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

Client: Hook nonblocking fsync into the write path of ll_preadv_pwritev

6809ff8

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

libcephfs: Option to write + fsync via ceph_ll_nonblocking_readv_writev

201a355

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

client test: Add fsync to ll_preadv_pwritev test

13162f6

Signed-off-by: Frank S. Filz <ffilzlnx@mindspring.com>

ffilz force-pushed the nonblocking-io branch from 80be983 to 13162f6 Compare July 24, 2023 18:55

vshankar merged commit e99c17f into ceph:main Jul 25, 2023

vshankar removed the ready-to-merge label Jul 25, 2023

lxbsz mentioned this pull request Jul 25, 2023

mds/client: check the cephx mds auth access in client side #48027

Merged

14 tasks

Conversation

ffilz commented Sep 9, 2022

Contribution Guidelines

Checklist

Uh oh!

github-actions bot commented Sep 23, 2022

Uh oh!

ljflores commented Nov 21, 2022

Uh oh!

ffilz commented Dec 6, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vshankar commented Mar 15, 2023

Uh oh!

github-actions bot commented Apr 13, 2023

Uh oh!

mattbenjamin commented May 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattbenjamin commented May 1, 2023

Uh oh!

ffilz commented May 3, 2023

Uh oh!

ffilz commented May 3, 2023

Uh oh!

ffilz commented Jul 14, 2023

Uh oh!

vshankar commented Jul 18, 2023

Uh oh!

vshankar commented Jul 18, 2023

Uh oh!

vshankar commented Jul 18, 2023

Uh oh!

vshankar left a comment

Choose a reason for hiding this comment

Uh oh!

vshankar commented Jul 19, 2023

Uh oh!

vshankar commented Jul 21, 2023

Uh oh!

vshankar commented Jul 21, 2023

Uh oh!

vshankar commented Jul 21, 2023

Uh oh!

vshankar commented Jul 24, 2023

Uh oh!

vshankar commented Jul 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mattbenjamin commented May 1, 2023 •

edited

Loading