Skip to content

Coroutines after Reef#49737

Closed
adamemerson wants to merge 65 commits intomainfrom
wip-coro-after-reef
Closed

Coroutines after Reef#49737
adamemerson wants to merge 65 commits intomainfrom
wip-coro-after-reef

Conversation

@adamemerson
Copy link
Contributor

@adamemerson adamemerson commented Jan 13, 2023 •
edited
Loading

A topic branch for C++20 Coroutine work, to be merged once main is no longer in feature freeze

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@adamemerson adamemerson force-pushed the wip-coro-after-reef branch 3 times, most recently from 748ad53 to c42dec3 Compare January 23, 2023 17:48
@adamemerson adamemerson mentioned this pull request Jan 25, 2023
Closed
This was referenced Mar 1, 2023
Closed
Closed
Closed
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from 88c6d6d to 3edbc3f Compare March 3, 2023 20:54
@github-actions
Copy link

github-actions bot commented Mar 6, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@cbodley cbodley mentioned this pull request Mar 6, 2023
Closed
14 tasks
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from 3edbc3f to cb02065 Compare March 20, 2023 22:09
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from cb02065 to c9fd0c5 Compare April 4, 2023 18:14
@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from bd1de90 to ecb46f1 Compare May 22, 2023 17:50
@petrutlucian94
Copy link
Contributor

petrutlucian94 commented May 23, 2023

The Windows job fails as we're attempting to patch some *.py files that have been removed in the latest Boost version:

ceph/win32_deps_build.sh

Lines 160 to 164 in 0fb0dc4

sed -i 's/mthreads/pthreads/g' ./tools/build/src/tools/gcc.py
sed -i 's/mthreads/pthreads/g' ./tools/build/src/tools/gcc.jam
sed -i 's/pthreads/mthreads/g' ./tools/build/src/tools/gcc.py
sed -i 's/pthreads/mthreads/g' ./tools/build/src/tools/gcc.jam

FWIW, I've prepared a PR that adds mingw-llvm support which also removes those lines: https://github.com/ceph/ceph/pull/51197/files#diff-b38927b949ef45961641fa185a21f84457428215f75afd7e860c9eb6bac5b9dcL161-L165

@adamemerson adamemerson force-pushed the wip-coro-after-reef branch from ecb46f1 to 42f22d8 Compare May 31, 2023 00:32
@github-actions
Copy link

github-actions bot commented Jun 21, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

adamemerson and others added 24 commits September 14, 2023 17:48
@adamemerson
neorados/cls: Client for version objclass
082e7b4 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@cbodley @adamemerson
common/async: handler wrappers forward associated cancellation_slot
7145dfc 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add service template for execution_context shutdown
53427df 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_throttle for bounded concurrency with c++20 coro…
1c39ad5 
…utines

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson
cls/log: Switch from std::list to std::vector
9ef642d 
We should not be using std::list everywhere, and this is an excellent
time to switch.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: Add non-default constructors to cls_log_entry
a103d12 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: Switch from utime_t to ceph::real_time
8158799 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
cls/log: C++ namespaces exist
481cb19 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
neorados/cls: cls::exec handles tuples properly
c53fc9f 
If our decoder function returns a tuple of multiple values, flatten it
so our signature is `void(error_code, T, U, V)` not
`void(error_code, std::tuple<T, U, V>)`.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
neorados/cls/version: Don't swallow exceptions
93a4646 
Since they can be reported now, report them

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
neorados/cls: Client for log objclass
6f70145 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@cbodley @adamemerson
common/async: move service out of detail
4716125 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_waiter class template
3e6c497 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: co_throttle_impl uses co_waiter
eface54 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
rgw/sync: add with_lease() with polymorphic LockClient
e1b9b6c 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
rgw/sync: wrap lease renewal exceptions in lease_aborted
d72e2e4 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: remove error_codes from co_throttle interface
6b84b7b 
in the initial design of co_throttle described in
#49720, the cancel_on_error option only
applied to errors from awaitable<error_code> but not to exceptions from
awaitable<void> coroutines

with the decision to use exceptions as the default method of error
handling in rgw multisite, this design choice no longer makes sense.
i've removed the error_code overloads entirely, and changed the
exception handling logic to match the previous behavior for error codes

the unit tests were rewritten with co_waiter instead of timers to make
them deterministic and faster. co_waiter's cancellation behavior
exposed some issues where the cancellation signal could cause the
completions to recurse, so on_complete() was restructured to tolerate
that

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add co_waiter::op_cancellation ctor for clang
4392c14 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add spawn_group template for fork-join parallelism
5b0e8f2 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@cbodley @adamemerson
common/async: add parallel_for_each() algorithm
5b9b2c0 
Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson
common/async: io_context_pool constructor takes int64_t
8ea3dfe 
Just because this is what Ceph's config uses and it saves a narrowing
conversion. If we want to set a max value on the thread count, we
should do it in config.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: SAL drivers take boost::asio::io_context
e174500 
`RadosDriver` needs it, and since SAL generally uses `optional_yield`,
other stores are going to need it to implement that sensibly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
ceph/async: io_context_pool constructor/start takes init function
673d7fc 
Add overloads of the constructor and `start` that take an init
function, for setting up the thread before dropping into
`io_context::run`.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Asio frontend shares io_context with the rest of RGW
9f5fc1f 
Pull the `io_context` and threads out of `AsioFrontend`, pass in a
reference to `io_context_pool` so it can be shut down at `AsioFrontend::join`

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@github-actions
Copy link

github-actions bot commented Oct 9, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@github-actions
Copy link

github-actions bot commented Dec 8, 2023

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

build/ops cephfs Ceph File System common core mon needs-rebase rbd rgw stale tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@adamemerson @petrutlucian94 @cbodley