Skip to content

storage: Fix flaky TestReplicaNotLeaseHolderError#25676

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
a-robinson:TestReplicaNotLeaseHolderError
May 23, 2018
Merged

storage: Fix flaky TestReplicaNotLeaseHolderError#25676
craig[bot] merged 1 commit intocockroachdb:masterfrom
a-robinson:TestReplicaNotLeaseHolderError

Conversation

@a-robinson
Copy link
Copy Markdown
Contributor

@a-robinson a-robinson commented May 18, 2018

Caused by the proactive renewal of expiration-based leases (see #25322
and #25625 for more detail).

Fixes #25731

Release note: None

I'm not sure how many more of these flakes to expect over the next couple weeks. I'm currently running stressrace on all of pkg/storage on a gceworker to try and ferret them out if there are any more, but there's no guarantee that'll find all of them.

@a-robinson a-robinson requested review from a team and bdarnell May 18, 2018 20:40
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@a-robinson
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented May 19, 2018

Build failed

@tbg
Copy link
Copy Markdown
Member

tbg commented May 19, 2018

DrainTransferWithExtension failed :(

Caused by the proactive renewal of expiration-based leases (see cockroachdb#25322
and cockroachdb#25625 for more detail).

Fixes cockroachdb#25672

Release note: None
@a-robinson a-robinson force-pushed the TestReplicaNotLeaseHolderError branch from 83ec2fd to 9706c03 Compare May 23, 2018 16:13
@a-robinson
Copy link
Copy Markdown
Contributor Author

DrainTransferWithExtension failed :(

That failure is very odd. It looks like nothing ever happened because node liveness heartbeats kept timing out. I'm not sure it's worth looking into unless it happens again, which it doesn't appear to have in teamcity lately.

@a-robinson
Copy link
Copy Markdown
Contributor Author

bors r+

craig bot pushed a commit that referenced this pull request May 23, 2018
25676: storage: Fix flaky TestReplicaNotLeaseHolderError r=a-robinson a=a-robinson

Caused by the proactive renewal of expiration-based leases (see #25322
and #25625 for more detail).

Fixes #25731

Release note: None

I'm not sure how many more of these flakes to expect over the next couple weeks. I'm currently running `stressrace` on all of `pkg/storage` on a gceworker to try and ferret them out if there are any more, but there's no guarantee that'll find all of them.

25816: ui: don't show login indicator unless login is enabled r=couchand a=couchand

In #25005 I accidentally let the login indicator leak even if the environment
variable wasn't set.  This corrects that issue.

Release note: None
Fixes: #25843

25853: dep: Bump grpc-go to include perf improvements and data race fix r=a-robinson a=a-robinson

Un-reverts #24883 to include grpc/grpc-go#2074

This also removed hdrhistogram-writer because it apparently isn't used
anymore.

Release note: None

Maybe we want to wait until they put this into a more formal release SHA, though.

Co-authored-by: Alex Robinson <alexdwanerobinson@gmail.com>
Co-authored-by: Andrew Couch <andrew@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented May 23, 2018

Build succeeded

@craig craig bot merged commit 9706c03 into cockroachdb:master May 23, 2018
craig bot pushed a commit that referenced this pull request Aug 30, 2018
29296: backport-2.0: Fix test flakes caused by automatic lease renewal r=a-robinson a=a-robinson

Backport:
  * 1/1 commits from "storage: Adding testing knob to disable automatic lease renewals" (#25625)
  * 1/1 commits from "storage: Fix flaky TestReplicaNotLeaseHolderError" (#25676)
  * 1/1 commits from "storage: Deflake TestReplicaLease" (#25781)

Please see individual PRs for details.

/cc @cockroachdb/release

Fixes #29189. The flakiness was introduced to the 2.0 branch by #28931.

Co-authored-by: Alex Robinson <alexdwanerobinson@gmail.com>
Co-authored-by: Ben Darnell <ben@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants