[GRPC] Optimize gRPC perf by passing by reference by karenyrx · Pull Request #18303 · opensearch-project/OpenSearch

karenyrx · 2025-05-15T01:00:43Z

Description

In the response-side of the transport-grpc plugin, response protobuf construction is primarily done by passing around copies of the partially constructed protobuf objects, populating them in various methods, and finally returning the combined protobuf object. This is much less efficient than passing all the protobuf builders by reference, which reduces the number of unecessary objects created and copied.

Implementing a pass-by-reference approach should reduce memory usage, garbage collection, CPU usage, and latency slightly, which was validated in a benchmark test below.

This PR also makes some minor refactoring changes and Javadoc fixes for readability and modularity.

Benchmarking results

Setup

Payload for the test was a MatchAll query (with source=true), returning 30k docs (~2.27MB).
Single data node in cluster with 8 cores, 64GB memory
Ran 3 trials and took the average.

Results

Testing (using Ballast, an internal perf testing tool) shows a small but consistent latency and CPU reduction with this change

Latency
CPU usage

Raw data

[Before] GRPC without change

Metric	Percentile	Trial 1	Trial 2	Trial 3	Average
Latency - First Request (ms)	P50	61.0	63.5	60.5	61.7
	P95	666.7	666.7	666.7	666.7
	P99	666.7	666.7	666.7	666.7
	P99.9	666.7	666.7	666.7	666.7
Latency - Subsequent Requests (ms)	P50	30.2	31.0	30.5	30.6
	P95	34.7	36.2	35.0	35.3
	P99	40.1	42.3	40.3	40.9
	P99.9	47.3	50.0	48.1	48.5
CPU Usage	Max	0.31	0.29	0.30	0.30
	Avg	0.27	0.26	0.27	0.27

[After] GRPC with pass by reference

Metric	Percentile	Trial 1	Trial 2	Trial 3	Average
Latency - First Request (ms)	P50	55.3	54.5	55.2	55.0
	P95	500.0	500.0	520.0	506.7
	P99	666.7	666.7	666.7	666.7
	P99.9	666.7	666.7	666.7	666.7
Latency - Subsequent Requests (ms)	P50	29.0	29.0	29.3	29.1
	P95	31.0	32.0	31.7	31.6
	P99	36.0	37.0	36.5	36.5
	P99.9	43.7	45.1	44.3	44.4
CPU Usage	Max	0.28	0.27	0.28	0.28
	Avg	0.25	0.23	0.24	0.24

Conclusions

Latency reduction: GRPC with pass reference has consistently lower latency for both first and subsequent requests compared to GRPC no change.
CPU Usage reduction: GRPC with pass reference has lower max and average CPU usage compared to GRPC no change.

Related Issues

Partly resolves #18291

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-05-15T01:09:45Z

❌ Gradle check result for 835b0df: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/common/ObjectMapProtoUtils.java

...in/java/org/opensearch/plugin/transport/grpc/proto/response/search/SearchHitsProtoUtils.java

github-actions · 2025-05-15T01:59:10Z

❌ Gradle check result for 8114025: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Karen Xu <karenxyr@gmail.com>

github-actions · 2025-05-15T03:14:25Z

❕ Gradle check result for 444f1ec: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2025-05-15T03:14:46Z

Codecov Report

Attention: Patch coverage is 85.18519% with 24 lines in your changes missing coverage. Please review.

Project coverage is 72.64%. Comparing base (aec3fe9) to head (9d3db04).
Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...rpc/proto/response/search/SearchHitProtoUtils.java	80.00%	9 Missing and 1 partial ⚠️
...oto/response/document/get/GetResultProtoUtils.java	46.66%	7 Missing and 1 partial ⚠️
...ponse/document/common/DocumentFieldProtoUtils.java	0.00%	4 Missing ⚠️
...pc/proto/response/common/FieldValueProtoUtils.java	90.90%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18303      +/-   ##
============================================
+ Coverage     72.52%   72.64%   +0.11%     
- Complexity    67496    67615     +119     
============================================
  Files          5495     5495              
  Lines        311433   311489      +56     
  Branches      45249    45223      -26     
============================================
+ Hits         225867   226275     +408     
+ Misses        67179    66806     -373     
- Partials      18387    18408      +21

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

...in/java/org/opensearch/plugin/transport/grpc/proto/response/common/FieldValueProtoUtils.java

...opensearch/plugin/transport/grpc/proto/response/document/common/DocumentFieldProtoUtils.java

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/search/SearchHitProtoUtils.java

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/common/ObjectMapProtoUtils.java

Signed-off-by: Karen Xu <karenxyr@gmail.com>

github-actions · 2025-05-23T00:06:37Z

✅ Gradle check result for 9ee0743: SUCCESS

Signed-off-by: Andrew Ross <andrross@amazon.com>

...in/java/org/opensearch/plugin/transport/grpc/proto/response/common/FieldValueProtoUtils.java

andrross · 2025-05-23T19:58:52Z

I rebased and fixed the changelog conflict. This is ready to merge once all tests pass.

github-actions · 2025-05-23T20:52:52Z

❌ Gradle check result for 9d3db04: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-05-27T14:54:21Z

✅ Gradle check result for 9d3db04: SUCCESS

…#18303) * [GRPC] Optimize gRPC perf by passing by reference Signed-off-by: Karen Xu <karenxyr@gmail.com> * Use pattern matching with switch expression Signed-off-by: Karen Xu <karenxyr@gmail.com> --------- Signed-off-by: Karen Xu <karenxyr@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com>

…#18303) * [GRPC] Optimize gRPC perf by passing by reference Signed-off-by: Karen Xu <karenxyr@gmail.com> * Use pattern matching with switch expression Signed-off-by: Karen Xu <karenxyr@gmail.com> --------- Signed-off-by: Karen Xu <karenxyr@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com>Signed-off-by: TJ Neuenfeldt <tjneu@amazon.com>

…#18303) * [GRPC] Optimize gRPC perf by passing by reference Signed-off-by: Karen Xu <karenxyr@gmail.com> * Use pattern matching with switch expression Signed-off-by: Karen Xu <karenxyr@gmail.com> --------- Signed-off-by: Karen Xu <karenxyr@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Andrew Ross <andrross@amazon.com>

github-actions bot added enhancement Enhancement or improvement to existing feature or request Search:Performance labels May 15, 2025

amberzsy reviewed May 15, 2025

View reviewed changes

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/common/ObjectMapProtoUtils.java Outdated Show resolved Hide resolved

karenyrx force-pushed the test-pass-reference branch from 5cfc899 to a062e83 Compare May 15, 2025 01:36

amberzsy reviewed May 15, 2025

View reviewed changes

...in/java/org/opensearch/plugin/transport/grpc/proto/response/search/SearchHitsProtoUtils.java Show resolved Hide resolved

[GRPC] Optimize gRPC perf by passing by reference

444f1ec

Signed-off-by: Karen Xu <karenxyr@gmail.com>

karenyrx force-pushed the test-pass-reference branch from 8114025 to 444f1ec Compare May 15, 2025 02:06

karenyrx marked this pull request as ready for review May 15, 2025 03:16

karenyrx requested a review from a team as a code owner May 15, 2025 03:16

varunbharadwaj reviewed May 15, 2025

View reviewed changes

...in/java/org/opensearch/plugin/transport/grpc/proto/response/common/FieldValueProtoUtils.java Outdated Show resolved Hide resolved

varunbharadwaj approved these changes May 15, 2025

View reviewed changes

opensearch-ci-bot mentioned this pull request May 15, 2025

[AUTOCUT] Gradle Check Flaky Test Report for AzureBlobStoreRepositoryTests #14291

Open

yupeng9 reviewed May 15, 2025

View reviewed changes

...opensearch/plugin/transport/grpc/proto/response/document/common/DocumentFieldProtoUtils.java Show resolved Hide resolved

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/search/SearchHitProtoUtils.java Show resolved Hide resolved

msfroh reviewed May 15, 2025

View reviewed changes

...ain/java/org/opensearch/plugin/transport/grpc/proto/response/common/ObjectMapProtoUtils.java Outdated Show resolved Hide resolved

karenyrx added 2 commits May 22, 2025 22:52

Use pattern matching with switch expression

3df4626

Signed-off-by: Karen Xu <karenxyr@gmail.com>

Merge remote-tracking branch 'ossupstream/main' into test-pass-reference

9ee0743

karenyrx force-pushed the test-pass-reference branch from fd0e65d to 9ee0743 Compare May 22, 2025 22:53

yupeng9 approved these changes May 23, 2025

View reviewed changes

msfroh approved these changes May 23, 2025

View reviewed changes

Merge branch 'main' into test-pass-reference

9d3db04

Signed-off-by: Andrew Ross <andrross@amazon.com>

andrross approved these changes May 23, 2025

View reviewed changes

andrross reviewed May 23, 2025

View reviewed changes

...in/java/org/opensearch/plugin/transport/grpc/proto/response/common/FieldValueProtoUtils.java Show resolved Hide resolved

andrross merged commit b82ba83 into opensearch-project:main May 27, 2025
30 checks passed

This was referenced May 28, 2025

[AUTOCUT] Gradle Check Flaky Test Report for BlobStoreRepositoryRemoteIndexTests #18380

Open

[AUTOCUT] Gradle Check Flaky Test Report for FullClusterRestartIT #15813

Open

BrewTestBot mentioned this pull request Jun 24, 2025

opensearch 3.1.0 Homebrew/homebrew-core#228059

Merged

msfroh mentioned this pull request Oct 7, 2025

Add varunbharadwaj as a maintainer #19557

Merged

This was referenced Jan 10, 2026

docs: add gRPC transport report for v3.1.0 tkykenmt/opensearch-feature-explorer#943

Merged

[feature] gRPC Transport tkykenmt/opensearch-feature-explorer#916

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GRPC] Optimize gRPC perf by passing by reference#18303

[GRPC] Optimize gRPC perf by passing by reference#18303
andrross merged 4 commits intoopensearch-project:mainfrom
karenyrx:test-pass-reference

karenyrx commented May 15, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

codecov bot commented May 15, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

Uh oh!

andrross commented May 23, 2025

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

github-actions bot commented May 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

karenyrx commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Benchmarking results

Setup

Results

Raw data

[Before] GRPC without change

[After] GRPC with pass by reference

Conclusions

Related Issues

Check List

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

github-actions bot commented May 15, 2025

Uh oh!

codecov bot commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

Uh oh!

andrross commented May 23, 2025

Uh oh!

github-actions bot commented May 23, 2025

Uh oh!

github-actions bot commented May 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

karenyrx commented May 15, 2025 •

edited

Loading

codecov bot commented May 15, 2025 •

edited

Loading