This repository was archived by the owner on Sep 30, 2024. It is now read-only.
grpc: raise global default message size from 4 MB to 90 MB#55209
Merged
Conversation
See https://sourcegraph.slack.com/archives/C04HCK4K3DL/p1689969401765809 for more context. This raises our global maximum message size that gRPC servers and clients are allowed to receive to 90 MB (up from 4MB). 4MB megabytes seems too restrictive for many of our use cases, and many projects (in the Slack message) seem to also increase the minimum.
camdencheek
approved these changes
Jul 21, 2023
Contributor
|
Codenotify: Notifying subscribers in CODENOTIFY files for diff aba1d58...0c85c28.
|
Contributor
|
The backport to To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-5.1 5.1
# Navigate to the new working tree
cd .worktrees/backport-5.1
# Create a new branch
git switch --create backport-55209-to-5.1
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 bce2029f622b1e652fe0624971d40f30943bd830
# Push it to GitHub
git push --set-upstream origin backport-55209-to-5.1
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-5.1Then, create a pull request where the |
ggilmore
added a commit
that referenced
this pull request
Jul 24, 2023
(cherry picked from commit bce2029)
coury-clark
pushed a commit
that referenced
this pull request
Jul 24, 2023
…55233) (cherry picked from commit bce2029) This is a custom cherry-pick since https://github.com/sourcegraph/sourcegraph/pull/55209 's backport ran into merge conflicts: https://github.com/sourcegraph/sourcegraph/pull/55209#issuecomment-1646247112 ## Test plan CI
ggilmore
referenced
this pull request
Jul 25, 2023
Update changelog for https://github.com/sourcegraph/sourcegraph/pull/55130 and https://github.com/sourcegraph/sourcegraph/pull/55209 5.1 backports.
Merged
MaedahBatool
pushed a commit
that referenced
this pull request
Jul 28, 2023
camdencheek
referenced
this pull request
Aug 2, 2023
…sages sent by servers/clients (#55495) Follow up to https://github.com/sourcegraph/sourcegraph/pull/55209 and https://github.com/sourcegraph/sourcegraph/pull/55242. This PR adds interceptors that records Prometheus metrics that observe: - the individual size of each **sent** protobuf message by a server or client - the total amount data sent over the course a single RPC by a server (responses) or client (requests) This allows us to track the total amount of a data returned by any of our RPCs. In some cases, this can reveal opportunities for future performance / stability improvements (Example: symbols' [LocalCodeIntel method returning ~gigabyte sized responses that has to be held all at once in memory](https://github.com/sourcegraph/sourcegraph/pull/55242)). This PR also provides new grafana dashboards that track this metric for every gRPC service. See below for a screenshot of what this looks like when I run the symbols service locally. Co-authored-by: Geoffrey Gilmore <geoffrey@sourcegraph.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See https://sourcegraph.slack.com/archives/C04HCK4K3DL/p1689969401765809 for more context.
This raises our global maximum message size that gRPC servers and clients are allowed to receive to 90 MB (up from 4MB). 4MB megabytes seems too restrictive for many of our use cases, and many projects (in the Slack message) seem to also increase their minimum. https://sourcegraph.com/search?q=context:global+MaxCallRecvMsgSize+-f:vendor+-r:grpc/grpc-go&patternType=standard&sm=1&groupBy=repo
However, at the same time we should strive to keep our message sizes as small as possible. I will prioritize adding additional Prometheus metrics to track this next week.
Note: Our existing REST code effectively has no cap on message sizes, so we're still no worse off than we were before.
Test plan
CI