Skip to content

roachtest: enable txn heartbeat loops for 1PC txns in kv/contention/nodes=4#45568

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/launchHB
Mar 3, 2020
Merged

roachtest: enable txn heartbeat loops for 1PC txns in kv/contention/nodes=4#45568
craig[bot] merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/launchHB

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Mar 2, 2020

I noticed when debugging issues in #45482 that unhandled deadlocks occasionally
resolved themselves because txns would eventually time out. This was because
we don't start the txn heartbeat loop for 1PC txns. In this kind of test, we
want any unhandled deadlocks to be as loud as possible, so just like we set
a very long txn expiration, we also enable the txn heartbeat loop for all
txns, even those that we expect will be 1PC.

This commit also drops the kv.lock_table.deadlock_detection_push_delay down
for the test, since it's already touching this code.

@nvb nvb requested a review from andreimatei March 2, 2020 04:31
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@andreimatei andreimatei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei and @nvanbenschoten)


pkg/cmd/roachtest/kv.go, line 225 at r1 (raw file):

			// Drop the deadlock detection delay because the test creates a
			// large number transaction deadlocks.
			if _, err := conn.Exec(`

I think there's a way to check the cluster version instead of swallowing failures

Copy link
Copy Markdown
Contributor Author

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andreimatei)


pkg/cmd/roachtest/kv.go, line 225 at r1 (raw file):

Previously, andreimatei (Andrei Matei) wrote…

I think there's a way to check the cluster version instead of swallowing failures

Yeah, that's another option. We already have this pattern all over the place though and I don't want to get into the game of guessing what the right cluster version is, so I'm going to stick with this.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 2, 2020

Canceled (will resume)

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 2, 2020

Build failed (retrying...)

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 3, 2020

bors r+

…odes=4

I noticed when debugging issues in cockroachdb#45482 that unhandled deadlocks occasionally
resolved themselves because txns would eventually time out. This was because
we don't start the txn heartbeat loop for 1PC txns. In this kind of test, we
want any unhandled deadlocks to be as loud as possible, so just like we set
a very long txn expiration, we also enable the txn heartbeat loop for all
txns, even those that we expect will be 1PC.

This commit also drops the kv.lock_table.deadlock_detection_push_delay down
for the test, since it's already touching this code.
@nvb nvb force-pushed the nvanbenschoten/launchHB branch from 9122438 to 898383f Compare March 3, 2020 03:20
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Mar 3, 2020

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 3, 2020

Build failed (retrying...)

@yuzefovich
Copy link
Copy Markdown
Member

Bors crashed because of my PR, so:

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 3, 2020

Already running a review

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Mar 3, 2020

Build succeeded

@craig craig bot merged commit 5f9a71a into cockroachdb:master Mar 3, 2020
@nvb nvb deleted the nvanbenschoten/launchHB branch March 3, 2020 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants