Skip to content

rgw: beast frontend uses 512k mprotected coroutine stacks#31580

Merged
mattbenjamin merged 8 commits intoceph:masterfrom
cbodley:wip-rgw-spawn
Jan 13, 2020
Merged

rgw: beast frontend uses 512k mprotected coroutine stacks#31580
mattbenjamin merged 8 commits intoceph:masterfrom
cbodley:wip-rgw-spawn

Conversation

@cbodley
Copy link
Contributor

@cbodley cbodley commented Nov 12, 2019

adds a submodule for https://github.com/cbodley/spawn, a header-only fork of boost::asio::spawn(). the beast frontend uses this to customize the coroutine stack allocator in order to use mmap/mprotect to detect overflows. the default stack size is raised from 128K to 512K

depends on commits from #31567 so that cls_otp doesn't also need to link against spawn and boost::context

TODO:

fixes: https://tracker.ceph.com/issues/43739

@cbodley cbodley added the rgw label Nov 12, 2019
@cbodley cbodley force-pushed the wip-rgw-spawn branch 3 times, most recently from 7aeae74 to bb070cd Compare December 6, 2019 14:37
@cbodley cbodley requested a review from adamemerson December 6, 2019 15:28
@cbodley
Copy link
Contributor Author

cbodley commented Dec 6, 2019

cc @galsalomon66

@cbodley cbodley requested a review from dang December 6, 2019 15:37
Copy link
Contributor

@dang dang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

@cbodley cbodley changed the title [RFC] rgw: beast frontend uses 512k mprotected coroutine stacks rgw: beast frontend uses 512k mprotected coroutine stacks Dec 6, 2019
@cbodley cbodley force-pushed the wip-rgw-spawn branch 2 times, most recently from 17a5c5c to f9b481e Compare December 19, 2019 17:50
cbodley and others added 8 commits January 8, 2020 10:35
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
BuildBoost.cmake (used when we're building the submodule) doesn't
provide parity with FindBoost.cmake (used with system Boost).
Specifically, it doesn't set the _FOUND variables for the various
components, making it hard to depend on finding those features.

Set Boost_<component>_FOUND for all the components we're building in
BuildBoost.cmake to make using these variables possible.

Signed-off-by: Daniel Gryniewicz <dang@redhat.com>
@cbodley
Copy link
Contributor Author

cbodley commented Jan 8, 2020

rebased over recent cmake changes

@cbodley
Copy link
Contributor Author

cbodley commented Jan 13, 2020

passed qa in http://pulpito.ceph.com/cbodley-2020-01-13_16:53:45-rgw-wip-cbodley-testing-distro-basic-smithi/

will need a submodule override to merge

@mattbenjamin mattbenjamin merged commit 20618a7 into ceph:master Jan 13, 2020
@cbodley cbodley deleted the wip-rgw-spawn branch January 13, 2020 19:47
cbodley referenced this pull request in cbodley/ceph Nov 27, 2020
Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Feb 15, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Feb 15, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Feb 15, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Feb 15, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Feb 20, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Apr 23, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Apr 24, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request Apr 24, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
cbodley added a commit to cbodley/ceph that referenced this pull request May 13, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
adamemerson pushed a commit to adamemerson/ceph that referenced this pull request May 16, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
adamemerson pushed a commit to adamemerson/ceph that referenced this pull request May 21, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
vshankar pushed a commit to vshankar/ceph that referenced this pull request May 22, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
guits pushed a commit to guits/ceph that referenced this pull request May 23, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
liangmingyuanneo pushed a commit to liangmingyuanneo/ceph that referenced this pull request Oct 5, 2024
a fork of boost::asio::spawn() was introduced in 2020 with spawn::spawn() from ceph#31580. this fork enabled rgw to customize how the coroutine stacks are allocated in order to avoid stack overflows in frontend request coroutines. this customization was based on a StackAllocator concept from the boost::context library

in boost 1.80, that same StackAllocator overload was added to boost::asio::spawn(), along with other improvements like per-op cancellation. now that boost has everything we need, switch back and drop the spawn submodule

this required switching a lot of async functions from async_completion<> to async_initiate<>. similar changes were necessary to enable the c++20 coroutine token boost::asio::use_awaitable

Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 9dd892e)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants