Skip to content

roachtest: fix release-20.1 roachtests failing due to double-init#52040

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
irfansharif:200728.roachprod-init-20.1
Jul 29, 2020
Merged

roachtest: fix release-20.1 roachtests failing due to double-init#52040
craig[bot] merged 1 commit intocockroachdb:masterfrom
irfansharif:200728.roachprod-init-20.1

Conversation

@irfansharif
Copy link
Copy Markdown
Contributor

Fixes #51965 (and all referencing issues).

Roachprod clusters running v20.1+ crdb nodes persist this
cluster-bootstrapped file on disk after explicitly bootstrapping the
cluster. Roachprod then uses the existence of this file to avoid doubly
bootstrapping the cluster.

Given #51897 remains unresolved, master-built roachprod is used to run
roachtests against the 20.1 branch. Some of those roachtests test
mixed-version clusters that start off at 19.2. Consequently, we manually
add this file where roachprod expects to find it for already-initialized
clusters. (This is a pretty gross hack, that we should address by
addressing #51897.)

Release note: None

@irfansharif irfansharif requested a review from nvb July 28, 2020 22:32
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@irfansharif irfansharif requested a review from jlinder July 28, 2020 22:32
Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @irfansharif, @jlinder, and @nvanbenschoten)


pkg/cmd/roachtest/main.go, line 164 at r1 (raw file):

	}

	// TODO(irfansharif): We could remove this up by directly running `cockroach

[nit] "remove this up"

Fixes cockroachdb#51965 (and all referencing issues).

Roachprod clusters running v20.1+ crdb nodes persist this
`cluster-bootstrapped` file on disk after explicitly bootstrapping the
cluster. Roachprod then uses the existence of this file to avoid doubly
bootstrapping the cluster.

Given cockroachdb#51897 remains unresolved, master-built roachprod is used to run
roachtests against the 20.1 branch. Some of those roachtests test
mixed-version clusters that start off at 19.2. Consequently, we manually
add this file where roachprod expects to find it for already-initialized
clusters. (This is a pretty gross hack, that we should address by
addressing cockroachdb#51897.)

Release note: None
@irfansharif irfansharif force-pushed the 200728.roachprod-init-20.1 branch from 4638e8f to d1ddaf5 Compare July 28, 2020 22:53
Copy link
Copy Markdown
Contributor Author

@irfansharif irfansharif left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @jlinder and @nvanbenschoten)


pkg/cmd/roachtest/main.go, line 164 at r1 (raw file):

Previously, RaduBerinde wrote…

[nit] "remove this up"

Done.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 28, 2020

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 29, 2020

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 29, 2020

Build failed:

@irfansharif
Copy link
Copy Markdown
Contributor Author

 Failed to publish artifacts: Failed to publish files: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/acceptance/multitenant/run_1/run_024231.443_n1_cockroach_mt.log (No such file or directory) 

I don't know what this is.

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 29, 2020

Build failed:

@RaduBerinde
Copy link
Copy Markdown
Member

Trying again, though this "Failed to publish files" issue seems persistent (I've seen it on a 20.1 PR as well)

bors r+

craig bot pushed a commit that referenced this pull request Jul 29, 2020
52040: roachtest: fix release-20.1 roachtests failing due to double-init r=RaduBerinde a=irfansharif

Fixes #51965 (and all referencing issues).

Roachprod clusters running v20.1+ crdb nodes persist this
`cluster-bootstrapped` file on disk after explicitly bootstrapping the
cluster. Roachprod then uses the existence of this file to avoid doubly
bootstrapping the cluster.

Given #51897 remains unresolved, master-built roachprod is used to run
roachtests against the 20.1 branch. Some of those roachtests test
mixed-version clusters that start off at 19.2. Consequently, we manually
add this file where roachprod expects to find it for already-initialized
clusters. (This is a pretty gross hack, that we should address by
addressing #51897.)

Release note: None

Co-authored-by: irfan sharif <irfanmahmoudsharif@gmail.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 29, 2020

Build failed:

@irfansharif
Copy link
Copy Markdown
Contributor Author

irfansharif commented Jul 29, 2020 via email

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 29, 2020

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

roachtest: release-20.1: double init due to restart of node 1

3 participants