Skip to content

*: skip flaky tests#99121

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
adityamaru:skip-flakes
Mar 21, 2023
Merged

*: skip flaky tests#99121
craig[bot] merged 1 commit intocockroachdb:masterfrom
adityamaru:skip-flakes

Conversation

@adityamaru
Copy link
Copy Markdown
Contributor

Informs: #98987
Informs: #98020
Informs: #98905
Informs: #98564
Informs: #98991

Release note: None
Epic: none

@adityamaru adityamaru requested review from a team and aadityasondhi March 21, 2023 14:44
@adityamaru adityamaru requested review from a team as code owners March 21, 2023 14:44
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Informs: cockroachdb#98987
Informs: cockroachdb#98020

Release note: None
Epic: none
@adityamaru adityamaru requested a review from a team as a code owner March 21, 2023 15:11
@adityamaru adityamaru requested review from smg260 and srosenberg and removed request for a team March 21, 2023 15:11
@adityamaru
Copy link
Copy Markdown
Contributor Author

bors r+ p=999 single on

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 21, 2023

Build succeeded:

@craig craig bot merged commit 5c48aa2 into cockroachdb:master Mar 21, 2023
@adityamaru adityamaru deleted the skip-flakes branch March 21, 2023 16:46
ajstorm added a commit to ajstorm/cockroach that referenced this pull request Mar 22, 2023
TestTenantUpgradeInterlock was skipped with cockroachdb#99121 because it was timing out
regularly. In response to the timeouts however, cockroachdb#98997 was merged to increase
the timeout duration for the test (since its run length is proportional to the
number of migrations in the release, which has been increasing). Unfortuntely,
our wires got crossed and the skip was introduced before the test had a chance
to run with the new timeout. cockroachdb#98987 hasn't shown a failure with the new timeout
length, so it may be the case that the longer timeout has resolved the problem.

Reenabling the test to see if it has better success under the new timeout
length.

Fixes: cockroachdb#98987
Epic: None
Release note: None
ajstorm added a commit to ajstorm/cockroach that referenced this pull request Mar 22, 2023
TestTenantUpgradeInterlock was skipped with cockroachdb#99121 because it was timing out
regularly. In response to the timeouts however, cockroachdb#98997 was merged to increase
the timeout duration for the test (since its run length is proportional to the
number of migrations in the release, which has been increasing). Unfortuntely,
our wires got crossed and the skip was introduced before the test had a chance
to run with the new timeout. cockroachdb#98987 hasn't shown a failure with the new timeout
length, so it may be the case that the longer timeout has resolved the problem.

Reenabling the test to see if it has better success under the new timeout
length.

Fixes: cockroachdb#98987
Epic: None
Release note: None
craig bot pushed a commit that referenced this pull request Mar 22, 2023
99216: multitenant: re-enable TestTenantUpgradeInterlock r=knz a=ajstorm

TestTenantUpgradeInterlock was skipped with #99121 because it was timing out regularly. In response to the timeouts however, #98997 was merged to increase the timeout duration for the test (since its run length is proportional to the number of migrations in the release, which has been increasing). Unfortuntely, our wires got crossed and the skip was introduced before the test had a chance to run with the new timeout. #98987 hasn't shown a failure with the new timeout length, so it may be the case that the longer timeout has resolved the problem.

Reenabling the test to see if it has better success under the new timeout length.

Fixes: #98987
Epic: None
Release note: None

Co-authored-by: Adam Storm <storm@cockroachlabs.com>
blathers-crl bot pushed a commit that referenced this pull request Mar 23, 2023
TestTenantUpgradeInterlock was skipped with #99121 because it was timing out
regularly. In response to the timeouts however, #98997 was merged to increase
the timeout duration for the test (since its run length is proportional to the
number of migrations in the release, which has been increasing). Unfortuntely,
our wires got crossed and the skip was introduced before the test had a chance
to run with the new timeout. #98987 hasn't shown a failure with the new timeout
length, so it may be the case that the longer timeout has resolved the problem.

Reenabling the test to see if it has better success under the new timeout
length.

Fixes: #98987
Epic: None
Release note: None
craig bot pushed a commit that referenced this pull request Mar 27, 2023
Part of #98020.

These tests were also skipped with datadriven language out in the
testdata files in #99121, so let's remove the ad-hoc skips here.

Confirming they're still skipped after this change:

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/regional_by_table
...
=== RUN   TestMultiRegionDataDriven/regional_by_table
    datadriven_test.go:129: [#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/regional_by_table (0.00s)
...
```

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/secondary_region
...
=== RUN   TestMultiRegionDataDriven/secondary_region
    datadriven_test.go:129: [#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/secondary_region (0.00s)
...
```

Release note: None
craig bot pushed a commit that referenced this pull request Mar 27, 2023
99484: ccl/multiregionccl: skip these tests just once r=matthewtodd a=matthewtodd

Part of #98020.

These tests were also skipped with datadriven language out in the testdata files in #99121, so let's remove the ad-hoc skips here.

Confirming they're still skipped after this change:

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/regional_by_table
...
=== RUN   TestMultiRegionDataDriven/regional_by_table
    datadriven_test.go:129: [#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/regional_by_table (0.00s)
...
```

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/secondary_region
...
=== RUN   TestMultiRegionDataDriven/secondary_region
    datadriven_test.go:129: [#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/secondary_region (0.00s)
...
```

Release note: None

Co-authored-by: Matthew Todd <todd@cockroachlabs.com>
aadityasondhi pushed a commit to aadityasondhi/cockroach that referenced this pull request Mar 28, 2023
Part of cockroachdb#98020.

These tests were also skipped with datadriven language out in the
testdata files in cockroachdb#99121, so let's remove the ad-hoc skips here.

Confirming they're still skipped after this change:

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/regional_by_table
...
=== RUN   TestMultiRegionDataDriven/regional_by_table
    datadriven_test.go:129: [cockroachdb#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/regional_by_table (0.00s)
...
```

```
…/cockroach (un-double-skip +) ./dev test -v pkg/ccl/multiregionccl/ -f=TestMultiRegionDataDriven/secondary_region
...
=== RUN   TestMultiRegionDataDriven/secondary_region
    datadriven_test.go:129: [cockroachdb#98020]
=== CONT  TestMultiRegionDataDriven
    datadriven_test.go:428: -- test log scope end --
--- PASS: TestMultiRegionDataDriven (0.00s)
    --- SKIP: TestMultiRegionDataDriven/secondary_region (0.00s)
...
```

Release note: None
srosenberg added a commit to srosenberg/cockroach that referenced this pull request Jun 27, 2023
TestSystemZoneConfigs was part of a batch of skipped tests [1].
It appears several tests in that batch flaked due to a
"slow quiesce" issue. The root cause for TestSystemZoneConfigs
flaking during that time remains unknown, yet the other tests
in the same batch have since been unskipped.

After stress-testing via,

```
./dev test pkg/kv/kvserver -f=TestSystemZoneConfigs --stress --stress-args '-maxruns 10000 -p 23' --timeout=10m
```
we surmise this test might have been indirectly deflaked.
Thus, we unskip it.

[1] cockroachdb#99121

Epic: none
Fixes: cockroachdb#98905

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants