Make beacon processor queue sizes dynamic#5573
Merged
mergify[bot] merged 4 commits intosigp:unstablefrom Jun 3, 2024
Merged
Conversation
paulhauner
approved these changes
Apr 22, 2024
Member
paulhauner
left a comment
There was a problem hiding this comment.
Looks good, very simple. I was thinking of dynamic as adjusting the queue size as the validator set grows but doing it once at startup is a big gain for very little complexity 👍 I don't think it's clear that fully dynamic is really worth the complexity.
I made a minor suggestion, feel free to take it or leave it.
dapplion
commented
Apr 22, 2024
0b8f4eb to
35ac9f4
Compare
Member
|
Nominating this for 5.2 as we are increasingly seeing queues filling up shortly after a restart |
michaelsproul
added a commit
that referenced
this pull request
May 31, 2024
Squashed commit of the following: commit 35ac9f4 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu May 30 13:15:50 2024 +0300 Review PR commit 7a4e44b Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:24:44 2024 +0900 lint commit 7590494 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:05:06 2024 +0900 Update tests commit 9460d58 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 15:17:27 2024 +0900 Make beacon processor queue sizes dynamic
Merged
michaelsproul
added a commit
that referenced
this pull request
Jun 3, 2024
Squashed commit of the following: commit 35ac9f4 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu May 30 13:15:50 2024 +0300 Review PR commit 7a4e44b Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:24:44 2024 +0900 lint commit 7590494 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:05:06 2024 +0900 Update tests commit 9460d58 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 15:17:27 2024 +0900 Make beacon processor queue sizes dynamic
jimmygchen
added a commit
that referenced
this pull request
Jun 3, 2024
Squashed commit of the following: commit 35ac9f4 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Thu May 30 13:15:50 2024 +0300 Review PR commit 7a4e44b Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:24:44 2024 +0900 lint commit 7590494 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 16:05:06 2024 +0900 Update tests commit 9460d58 Author: dapplion <35266934+dapplion@users.noreply.github.com> Date: Fri Apr 12 15:17:27 2024 +0900 Make beacon processor queue sizes dynamic
michaelsproul
approved these changes
Jun 3, 2024
Member
|
@Mergifyio queue |
✅ The pull request has been merged automaticallyDetailsThe pull request has been merged automatically at 2c971fa |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue Addressed
Some users have complained of their nodes rejecting attestations due to full queues. The current size of 16k was good before, but it's insufficient for the current mainnet's size. Instead of playing catch with these numbers, we can make them dynamic on the validator set size. The set size is not meant to change significantly between client restarts so sampling once at start-up should be sufficient.
Proposed Changes
Make beacon processor queue sizes dynamic, and some of its values variable on initial active validator set size. This change allows future testing of the processor where we want to make the queue very short to test their behavior.
NOTE: I have not changed any numbers except for
attestation_queue,unknown_block_attestation_queue. Happy to change or justify existing numbers if you have rationales.