dynamic_modules: scheduler ABI for generic non-blocking async operations #39765
Merged
mathetake merged 7 commits intoenvoyproxy:mainfrom Jun 17, 2025
Merged
dynamic_modules: scheduler ABI for generic non-blocking async operations #39765mathetake merged 7 commits intoenvoyproxy:mainfrom
mathetake merged 7 commits intoenvoyproxy:mainfrom
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit Message: dynamic_modules: scheduler ABI for generic non-blocking async operations
Additional Description:
This commits adds a generic "scheduler" ABI that allows modules to schedule a generic event trigged by the module. More specifically, this adds the capability to create a "scheduler" object per HTTP filter object (== per request) that can be safely sent across threads. It has the method to schedule a generic event invoked against the HTTP filter object on the correct worker thread, and that method can be called from any threads including the ones managed by a module. At Rust SDK level, the scheduler trait implements the Send trait hence can be sent to another thread to offload the processing while freeing the worker thread.
The following functions are added to the ABI:
This enables the whole bunch of new use cases. Most notably, now a module can implement their own External Processor, External Authorization, or Global Rate Limit filter style extensions. These require the blocking network calls in their own style so the scheduler allows it to avoid blocking the worker thread while doing the necessary work. One another example is an external-caching filter which talks to a database like DynamoDB, Redis, Memcached, S3, etc.
Here's the example Rust SDK usage which implements an external-caching filter (from the integration test case):
Risk Level: low (completely new optional code path)
Testing: done
Docs Changes: done
Release Notes: n/a
Platform Specific Features: n/a