roachtest: skip c2c/BulkOps and add the new c2c/BulkOps/SingleImport#105677
roachtest: skip c2c/BulkOps and add the new c2c/BulkOps/SingleImport#105677craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
|
This change is |
lidorcarmel
left a comment
There was a problem hiding this comment.
so, the original test is something that we want to work, right? then why relax it and also skip it? I'd say, either relax it to a level that it can be enabled, which means we'll get some test coverage for now, or, skip it until we improve performance. Looks like relaxing the test didn't do much so just skip it for now and leave it as is?
I'd hate to make the code more complex for no real reason.
83fde1f to
7e8d567
Compare
|
I see your point. I amended the commit to keep the original test as is (aside from reconfiguring the timeouts), and added a new "reduced form" test, which contains the simplest repro of #105676 and is also skipped. I think adding the simpler test is important, as it provides much easier way to debug and reproduce #105676 than the original test. The new test has a workload that takes 3 mins, the original workload takes an hour to run. |
7e8d567 to
f756feb
Compare
The roachtest c2c/BulkOps has been failing every night since it merged to master due to c2c's inability to keep up with the foreground source cluster workload. We had hoped to deflake the test with general c2c performance improvements, but have yet to get there. This patch skips the test and adds the c2c/BulkOps/SingleImport roachtest which reduces the complexity of the workload. In the original test, 2 ~30 GB imports would run + 2 import rollbacks after total ingestion, taking about an hour. This patch changes the workload to a single ~10GB import which takes about 3 minutes. Even with the simpler workload, the replication stream cannot catch up to the src tenant after an hour. To avoid nightly roachtest failures, both the original and new test will be skipped and the performance issues will get tracked in cockroachdb#105676. Fixes cockroachdb#98669 Informs cockroachdb#105676 Release note: None
f756feb to
e58eadd
Compare
|
TFTR! bors r=lidorcarmel |
|
Build succeeded: |
The roachtest c2c/BulkOps has been failing every night since it merged to
master due to c2c's inability to keep up with the foreground source cluster
workload. We had hoped to deflake the test with general c2c
performance improvements, but have yet to get there.
This patch skips the test and adds the c2c/BulkOps/SingleImport roachtest which
reduces the complexity of the workload. In the original test, 2 ~30 GB imports
would run + 2 import rollbacks after total ingestion, taking about an hour.
This patch changes the workload to a single ~10GB import which takes about 3
minutes. Even with the simpler workload, the replication stream cannot catch up
to the src tenant after an hour. To avoid nightly roachtest failures, both the
original and new test will be skipped and the performance issues will get
tracked in #105676.
Fixes #98669
Informs #105676
Release note: None