Skip to content

neorados/cls: FIFO client#49976

Closed
adamemerson wants to merge 3319 commits intoceph:wip-coro-after-reeffrom
adamemerson:wip-neorados-cls-fifo
Closed

neorados/cls: FIFO client#49976
adamemerson wants to merge 3319 commits intoceph:wip-coro-after-reeffrom
adamemerson:wip-neorados-cls-fifo

Conversation

@adamemerson
Copy link
Contributor

@adamemerson adamemerson commented Feb 2, 2023 •
edited
Loading

NeoRADOS CLS FIFO client.

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@adamemerson adamemerson requested a review from cbodley February 2, 2023 16:17
tchaikov
tchaikov reviewed Feb 3, 2023
View reviewed changes
@adamemerson adamemerson requested a review from a team as a code owner February 3, 2023 17:51
@github-actions
Copy link

github-actions bot commented Feb 3, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch from 271a902 to 166d4ba Compare February 3, 2023 18:00
@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch from 166d4ba to d8bd864 Compare February 3, 2023 19:50
cbodley
cbodley approved these changes Feb 6, 2023
View reviewed changes
Copy link
Contributor

@cbodley cbodley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good in general but hard to review in detail. we'll need to do some multisite workload testing to verify this, and that will depend on further async refactoring

@adamemerson
Copy link
Contributor Author

adamemerson commented Feb 13, 2023

I'm going to do another pass, especially since we now have only one version of everything, and thread the Error Injector through, so I can test the error paths, and if that works, I'll make a version integrated into RGW that we can do workload tests against.

@adamemerson adamemerson requested a review from a team as a code owner February 14, 2023 19:06
@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch from d8bd864 to 456f6fb Compare February 14, 2023 19:49
@github-actions
Copy link

github-actions bot commented Feb 28, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from 88c6d6d to 3edbc3f Compare March 3, 2023 20:54
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from 3edbc3f to cb02065 Compare March 20, 2023 22:09
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from cb02065 to c9fd0c5 Compare April 4, 2023 18:14
@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch from 456f6fb to e7601f4 Compare April 17, 2023 19:07
@github-actions github-actions bot added this to the reef milestone Apr 17, 2023
@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch 2 times, most recently from a7838d9 to 024be9e Compare April 28, 2023 18:27
@adamemerson adamemerson force-pushed the wip-neorados-cls-fifo branch from 024be9e to c5920d7 Compare May 22, 2023 17:42
adamemerson and others added 30 commits February 27, 2024 12:44
@adamemerson
neorados/cls: Client for version objclass
e79f53a 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@cbodley @adamemerson
common/async: handler wrappers forward associated cancellation_slot
b9a0380 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add service template for execution_context shutdown
36edb0e 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_throttle for bounded concurrency with c++20 coro…
eaa6069 
…utines

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson
cls/log: Switch from std::list to std::vector
faad0aa 
We should not be using std::list everywhere, and this is an excellent
time to switch.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: Add non-default constructors to cls_log_entry
0afb4a0 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: Switch from utime_t to ceph::real_time
26d8df9 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: C++ namespaces exist
f01db0e 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
neorados/cls: Client for log objclass
fd2f281 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@cbodley @adamemerson
common/async: move service out of detail
6342180 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_waiter class template
f730a93 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: co_throttle_impl uses co_waiter
92a8821 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
rgw/sync: add with_lease() with polymorphic LockClient
e15ff12 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
rgw/sync: wrap lease renewal exceptions in lease_aborted
df85fb8 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: remove error_codes from co_throttle interface
44e1302 
in the initial design of co_throttle described in
ceph#49720, the cancel_on_error option only
applied to errors from awaitable<error_code> but not to exceptions from
awaitable<void> coroutines

with the decision to use exceptions as the default method of error
handling in rgw multisite, this design choice no longer makes sense.
i've removed the error_code overloads entirely, and changed the
exception handling logic to match the previous behavior for error codes

the unit tests were rewritten with co_waiter instead of timers to make
them deterministic and faster. co_waiter's cancellation behavior
exposed some issues where the cancellation signal could cause the
completions to recurse, so on_complete() was restructured to tolerate
that

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_waiter::op_cancellation ctor for clang
6ab4d9e 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add spawn_group template for fork-join parallelism
2410f93 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add parallel_for_each() algorithm
86128be 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson
common/async: Bridging functions between async domains
325548c 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
common/async: Non-blocking condition variable
a553f5f 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
include/types: Add operator << for std::span
be5c079 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@cbodley @adamemerson
rgw/curl: async curl client
5b1be39 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
rgw/curl: switch to generic protocol sockets
629621c 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson
rgw: Version tracker for neorados
00e90e3 
Signed-off-by: Adam Emerson <aemerson@redhat.com>
@adamemerson
include/encoding: Add encodable and decodable concepts
e7de145 
Signed-off-by: Adam Emerson <aemerson@redhat.com>
@adamemerson
rgw: Add neorados utilities
4154b27 
Signed-off-by: Adam Emerson <aemerson@redhat.com>
@adamemerson
neorados/cls: Client for FIFO objclass
2744352 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Datalog takes driver, not handle.
d353a83 
Since we're plumbing the driver around and it's where our neorados
handle lives.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw/datalog: Get rid of unneeded set_zero
c647c28 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
radosgw: C++20 Coroutine Datalog Refactor
d282166 
Rewrite all of `RGWDataChangesLog` and supporting classes to use
non-blocking, C++20 coroutines.

Make interfaces for `optional_yield` and `librados::AioCompletion`.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tchaikov tchaikov tchaikov left review comments

@cbodley cbodley cbodley approved these changes

Assignees

No one assigned

Labels

api-change bluestore build/ops ceph-volume cephadm cephfs Ceph File System CI Continuous Integration common core crimson dashboard documentation mgr mon monitoring needs-rebase nfs orchestrator pybind rbd rgw rook telemetry tests

Projects

Ceph-Dashboard
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.