roachtest/cdc: add roachtest for per changefeed kafka quota config#118428
Closed
wenyihu6 wants to merge 5 commits intocockroachdb:masterfrom
Closed
roachtest/cdc: add roachtest for per changefeed kafka quota config#118428wenyihu6 wants to merge 5 commits intocockroachdb:masterfrom
wenyihu6 wants to merge 5 commits intocockroachdb:masterfrom
Conversation
Member
|
This change is |
cc44b3e to
edd9e94
Compare
62b30ff to
75c3be1
Compare
Now that this patch has been merged, sarama now acknowledges and reacts to kafka server's throttling messages by slowing down. To provide better observability into sarama code, this patch adds a metrics registry interceptor and a new metrics `changefeed.kafka_throttling_hist_nanos` which tracks time (in nanos) spent in sarama's throttling when cockroachdb exceed the kafka quota. Release note: changefeed.kafka_throttling_hist_nanos has now been added to metrics to monitor sarama throttling behavior resulting from exceeding kafka quota. Fixes: cockroachdb#117618
This patch adds a new roachtest to observe kafka throttling behaviour. It's a bit tricky to find the passing condition here since we are expecting throttling behaviour and would like to assert that the latency is above the target. Part of: cockroachdb#117618 Release note:none
0c8b4b7 to
e6ff5ee
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Splitting this PR out for adding a roachtest.