Skip to content

grpc: standardize the cache interface of grpc async client.#15745

Merged
jmarantz merged 62 commits intoenvoyproxy:mainfrom
chaoqin-li1123:abstract_async_client_cache
Jul 15, 2021
Merged

grpc: standardize the cache interface of grpc async client.#15745
jmarantz merged 62 commits intoenvoyproxy:mainfrom
chaoqin-li1123:abstract_async_client_cache

Conversation

@chaoqin-li1123
Copy link
Copy Markdown
Contributor

@chaoqin-li1123 chaoqin-li1123 commented Mar 29, 2021

Commit Message: Creating a grpc async client can be expensive. Caching and reusing grpc async client can save resources and improve stats consistency. This pr add a thread local cache of grpc async client as a member of grpc async client manager.
Additional Description:N/A
Risk Level:medium
Testing:add unit and integration tests
Docs Changes:N/A
Release Notes:
Platform Specific Features:
Fixes: #12598

chaoqin-li1123 added 9 commits March 2, 2021 21:55
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
chaoqin-li1123 added 5 commits March 29, 2021 15:17
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@chaoqin-li1123 chaoqin-li1123 changed the title [WIP]: standardize the cache interface of grpc async client. [grpc]: standardize the cache interface of grpc async client. Mar 29, 2021
Copy link
Copy Markdown
Member

@rojkov rojkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I'm not very familiar with the code, but left some driveby comments.

Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@chaoqin-li1123 chaoqin-li1123 requested a review from dio as a code owner March 31, 2021 15:29
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@htuch
Copy link
Copy Markdown
Member

htuch commented Apr 1, 2021

@adisuissa @dmitri-d I think you folks might be interested in taking a pass on this from a control plane perspective.

@jmarantz
Copy link
Copy Markdown
Contributor

This needs a non-google review I think. @lizan can you take a look?

lizan
lizan previously approved these changes Jul 13, 2021
@jmarantz
Copy link
Copy Markdown
Contributor

Thanks @lizan . @chaoqin-li1123 can you merge main?

@jmarantz
Copy link
Copy Markdown
Contributor

/wait

Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@chaoqin-li1123 chaoqin-li1123 dismissed stale reviews from lizan and htuch via 67741ec July 13, 2021 21:50
jmarantz
jmarantz previously approved these changes Jul 14, 2021
chaoqin-li1123 added 3 commits July 15, 2021 17:12
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@jmarantz jmarantz merged commit eded9bb into envoyproxy:main Jul 15, 2021
@mattklein123
Copy link
Copy Markdown
Member

@jmarantz @htuch @chaoqin-li1123 sorry late to this PR, but I see a lot of comments about cached vs. uncached. From a skim of the PR I'm not quite sure why some users were converted to cached and some were not. Can we at minimum backfill TODO comments or other comments on every callsite using uncached to explain why that is happening? From reading it quickly it's confusing. Thank you.

@chaoqin-li1123
Copy link
Copy Markdown
Contributor Author

@jmarantz @htuch @chaoqin-li1123 sorry late to this PR, but I see a lot of comments about cached vs. uncached. From a skim of the PR I'm not quite sure why some users were converted to cached and some were not. Can we at minimum backfill TODO comments or other comments on every callsite using uncached to explain why that is happening? From reading it quickly it's confusing. Thank you.

I think ultimately we want to use the cached version everywhere. But for now only the grpc client for ex_authz is default enabled, which is equivalent to the old behavior. We can open an issue as followup and clean this up in the following pull request.

@mattklein123
Copy link
Copy Markdown
Member

I think ultimately we want to use the cached version everywhere. But for now only the grpc client for ex_authz is default enabled, which is equivalent to the old behavior. We can open an issue as followup and clean this up in the following pull request.

Please go through and add comments/TODOs on each of the usages when you open the issue. Thank you!

@htuch
Copy link
Copy Markdown
Member

htuch commented Jul 19, 2021

Yes, this was also my question but I think the position of wanting to derisk this by starting with ext_authz is fine modulo filing the issues.

mum4k added a commit to mum4k/nighthawk that referenced this pull request Jul 20, 2021
- no changes in `.bazelrc`, `.bazelversion` or `ci/run_envoy_docker.sh`.
- reverting the `addIdleCallback` connection pool method back to `addDrainedCallback` as per envoyproxy/envoy#17319.
- changing method call `AsyncClientFactory::create()` to `AsyncClientFactory::createUncachedRawAsyncClient()` as per envoyproxy/envoy#15745.

Signed-off-by: Jakub Sobon <mumak@google.com>
mum4k added a commit to envoyproxy/nighthawk that referenced this pull request Jul 20, 2021
- no changes in `.bazelrc`, `.bazelversion` or `ci/run_envoy_docker.sh`.
- reverting the `addIdleCallback` connection pool method back to `addDrainedCallback` as per envoyproxy/envoy#17319.
- changing method call `AsyncClientFactory::create()` to `AsyncClientFactory::createUncachedRawAsyncClient()` as per envoyproxy/envoy#15745.

Signed-off-by: Jakub Sobon <mumak@google.com>
leyao-daily pushed a commit to leyao-daily/envoy that referenced this pull request Sep 30, 2021
…xy#15745)

Commit Message: Creating a grpc async client can be expensive. Caching and reusing grpc async client can save resources and improve stats consistency. This pr add a thread local cache of grpc async client as a member of grpc async client manager.
Additional Description:N/A
Risk Level:medium
Testing:add unit and integration tests
Docs Changes:N/A
Release Notes:
Platform Specific Features:
Fixes: envoyproxy#12598

Signed-off-by: chaoqin-li1123 <chaoqinli@google.com>
@alyssawilk
Copy link
Copy Markdown
Contributor

Checking back in as we cut the release, what's the plan for flipping this true?

@chaoqin-li1123
Copy link
Copy Markdown
Contributor Author

We finish the migration for ext_authz and ext_proc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Standardized abstraction for sharing gRPC async clients across filter instances of a given type/config

10 participants