backupccl: replace restore2TB and restoretpccInc tests#98072
Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom Mar 7, 2023
Merged
backupccl: replace restore2TB and restoretpccInc tests#98072craig[bot] merged 1 commit intocockroachdb:masterfrom
craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
Member
|
This change is |
lidorcarmel
reviewed
Mar 6, 2023
Contributor
lidorcarmel
left a comment
There was a problem hiding this comment.
overall makes sense, a few things:
- you're also dropping a tpcc roachperf, right? probably intentional, mention it in the pr/commit description?
- in roachperf we have 2 bank tests with 6 nodes, one has no data, I'm guessing it was removed from restore.go but not from roachperf? or something like that.. which brings me to the question - do you need to remove those from roachperf also?
- the 32 nodes bank test [1] shows a regression since last Dec, we're going to lose this info now. The newly added tests here don't backfill, so maybe we should keep those tests for a few more months to have an overlap with the new tests? WDYT?
[1] https://roachperf.crdb.dev/?filter=restore&view=restore2TB%2Fnodes%3D32&tab=gce
Collaborator
Author
to paraphrase your last question: " should we keep them around for a couple months for perf regression detection?"
|
Contributor
I think it is helpful to have it, it helps us understand that things regressed slowly in that time range. I know it's not the best test but it's info I'd like to have available for debugging. anyway, sounds like we'll have the history so we're good, thank you! |
lidorcarmel
approved these changes
Mar 6, 2023
This patch removes the restore2TB* roachtests which ran a 2TB bank restore to benchmark restore performance across a few hardware configurations. This patch also replaces the `restoreTPCCInc/nodes=10` test which tested our ability to handle a backup with a long chain. This patch also adds: 1. `restore/tpce/400GB/aws/nodes=4/cpus=16` to measure how per-node throughput scales when the per node vcpu count doubles relative to default. 2. `restore/tpce/400GB/aws/nodes=8/cpus=8` to measure how per-node throughput scales when the number of nodes doubles relative to default. 3. `restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8` to measure restore reliability and performance on 48 length long backup chain relative to default. A future patch will update the fixtures used in the restore node shutdown scripts, and add more perf based tests. Fixes cockroachdb#92699 Release note: None
0296cb2 to
b588f1f
Compare
Collaborator
Author
|
TFTR! bors r=lidorcarmel |
Contributor
|
Build succeeded: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch removes the restore2TB* roachtests which ran a 2TB bank restore to
benchmark restore performance across a few hardware configurations. This patch
also replaces the
restoreTPCCInc/nodes=10test which tested our ability tohandle a backup with a long chain.
This patch also adds:
restore/tpce/400GB/aws/nodes=4/cpus=16to measure how per-node throughputscales when the per node vcpu count doubles relative to default.
restore/tpce/400GB/aws/nodes=8/cpus=8to measure how per-node throughputscales when the number of nodes doubles relative to default.
restore/tpce/400GB/aws/backupsIncluded=48/nodes=4/cpus=8to measurerestore reliability and performance on 48 length long backup chain relative to
default.
A future patch will update the fixtures used in the restore node shutdown
scripts, and add more perf based tests.
Fixes #92699
Release note: None