Skip to content

*: upgrade to go 1.18.4#84590

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rickystewart:upgradego118
Jul 25, 2022
Merged

*: upgrade to go 1.18.4#84590
craig[bot] merged 1 commit intocockroachdb:masterfrom
rickystewart:upgradego118

Conversation

@rickystewart
Copy link
Copy Markdown
Collaborator

  • Adjust the Pebble tests to run in new version.
  • Adjust version in Docker image (source).
  • Adjust version in the TeamCity agent image (setup script)
  • Rebuild and push the Docker image (following Basic Process)
  • Download ALL the archives (.tar.gz, .zip) for the new Go version from https://golang.org/dl/ and mirror them in the public-bazel-artifacts bucket in the Bazel artifacts project in GCP (sub-directory go, next to the other Go SDK's).
  • Bump the version in WORKSPACE under go_download_sdk. You may need to bump rules_go. Also edit the filenames listed in sdks and update all the hashes to match what you mirrored in the step above.
  • Run ./dev generate bazel to refresh distdir_files.bzl, then bazel fetch @distdir//:archives to ensure you've updated all hashes to the correct value.
  • Bump the version in builder.sh accordingly (source).
  • Bump the version in go-version-check.sh (source), unless bumping to a new patch release.
  • Bump the go version in go.mod. You may also need to rerun make vendor_rebuild if vendoring has changed.
  • Bump the default installed version of Go in bootstrap-debian.sh (source).
  • Replace other mentions of the older version of go (grep for golang:<old_version> and go<old_version>).
  • Update the builder.dockerImage parameter in the TeamCity Cockroach and Internal projects.
  • Ask the Developer Infrastructure team to deploy new TeamCity agent images according to packer/README.md

Release note (build change): Upgrade to golang 1.18.4

@rickystewart rickystewart requested a review from a team as a code owner July 18, 2022 16:31
@rickystewart rickystewart requested a review from a team July 18, 2022 16:31
@rickystewart rickystewart requested a review from a team as a code owner July 18, 2022 16:31
@rickystewart rickystewart requested review from tbg and removed request for a team July 18, 2022 16:31
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rickystewart rickystewart force-pushed the upgradego118 branch 5 times, most recently from 2040fa5 to b005b60 Compare July 20, 2022 21:55
@rickystewart rickystewart requested a review from a team as a code owner July 20, 2022 21:55
@rickystewart rickystewart requested a review from a team July 20, 2022 21:55
@rickystewart rickystewart requested review from a team as code owners July 20, 2022 21:55
@rickystewart rickystewart requested review from a team July 20, 2022 21:55
@rickystewart rickystewart requested a review from a team as a code owner July 20, 2022 21:55
@rickystewart rickystewart requested review from HonoreDB and msbutler and removed request for a team July 20, 2022 21:55
Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 58 of 58 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB, @msbutler, @rickystewart, and @tbg)


pkg/ccl/changefeedccl/changefeed_stmt.go line 978 at r1 (raw file):

				// due to a transient, retryable error.
				err = jobs.MarkAsRetryJobError(err)
				lastRunStatusUpdate = b.setJobRunningStatus(ctx, lastRunStatusUpdate, "retryable flow error: %s", err)  //lint:ignore SA4006

This looks like an outright bug. I'd remove the lint ignore directive and ask @HonoreDB what's up with it.


pkg/cli/debug_merge_logs.go line 341 at r1 (raw file):

	}
	//lint:ignore SA4006
	to = to.Truncate(time.Second) // log files only have second resolution

The fact that to is not used any more is either a bug (the code below should use it), or a remnant from a previous version.
In any case, a linter ignore here is not desirable.
@abarganier what should we do here?


pkg/kv/kvserver/deprecated_store_rebalancer.go line 124 at r1 (raw file):

			}

			filteredStoreList := storeList.ExcludeInvalid(conf.Constraints)

wowzer. This also looks like a bug. The removal you just made doesn't look like the right fix? I think we need to combine them somehow.
Even though the file name says "deprecated" I believe this code is still in use.

@aayushshah15 what do you think?


pkg/kv/kvserver/store_rebalancer.go line 461 at r1 (raw file):

		}

		filteredStoreList := allStoresList.ExcludeInvalid(conf.Constraints)

ditto


pkg/sql/colexec/columnarizer.go line 218 at r1 (raw file):

		newRows := make(rowenc.EncDatumRows, c.batch.Capacity())
		_ = newRows[len(oldRows)]
		//lint:ignore S1001

Maybe add a comment to explain we're avoiding copy because we need gcassert below.

( @yuzefovich could we use copy here? That's what the compiler is trying to tell us. )


pkg/sql/opt/cat/zone.go line 308 at r1 (raw file):

	voterChild := zoneChild
	if zone.VoterConstraintsCount() > 1 {
		voterChild = voterChild.Childf("voter replica constraints")  //lint:ignore SA4006

This also looks suspicious. I don't think we should ignore this linter warning.
@mgartner @rytaft do you want to have a look?


pkg/sql/opt/memo/statistics_builder.go line 4122 at r1 (raw file):

	for f := 0; f < len(h.filters); f++ {
		conjunctSelectivity = props.OneSelectivity

Here i'm also not what's up - why are we not looking at the conjunct selectivity any more?

cc @rytaft @mgartner


pkg/sql/opt/xform/scan_funcs.go line 320 at r1 (raw file):

	// let's not hide the problem by still generating a locality-optimized search
	// that doesn't fully cover local spans.
	//lint:ignore S1009

It's possible we do care about the difference between a nil slice reference and a zero-sized non-nil slice here, but I'm not sure.
Perhaps the code can indeed be simplified. If we keep it, we need an explanatory comment to motivate the linter ignore.

@mgartner can you provide guidance?

@rickystewart
Copy link
Copy Markdown
Collaborator Author

I've already contacted a couple people about some of these linter failures (but not all of them).

@rytaft addressed the problem with zone.go in #84768.

@ajwerner addressed the problem with debug_merge_logs.go in #84769.

I reached out to @aayushshah15 about the store_rebalancer issues but did not get a response yet.

Copy link
Copy Markdown
Contributor

@miretskiy miretskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB, @msbutler, @rickystewart, and @tbg)


pkg/ccl/changefeedccl/changefeed_stmt.go line 978 at r1 (raw file):

Previously, knz (kena) wrote…

This looks like an outright bug. I'd remove the lint ignore directive and ask @HonoreDB what's up with it.

I don't know;... it's True that we return below; but normally lastRunStatusUpdate is used, and is updated in a loop;
I suppose we could say "_ = b.setJobRunningStatus"... But is it that much better? It's definitely not a bug.

Copy link
Copy Markdown
Contributor

@miretskiy miretskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB, @msbutler, @rickystewart, and @tbg)


pkg/ccl/changefeedccl/changefeed_stmt.go line 978 at r1 (raw file):

Previously, miretskiy (Yevgeniy Miretskiy) wrote…

I don't know;... it's True that we return below; but normally lastRunStatusUpdate is used, and is updated in a loop;
I suppose we could say "_ = b.setJobRunningStatus"... But is it that much better? It's definitely not a bug.

Also, ignoring return values.. perhaps it's a thing, but I personally really don't like it.

@HonoreDB
Copy link
Copy Markdown
Contributor

pkg/ccl/changefeedccl/changefeed_stmt.go line 978 at r1 (raw file):

Previously, miretskiy (Yevgeniy Miretskiy) wrote…

Also, ignoring return values.. perhaps it's a thing, but I personally really don't like it.

Agree that it's not a bug. It's confusing to ignore the return value but arguably it'd be worse to have two different methods that do the same thing but one returns nothing.

@rickystewart
Copy link
Copy Markdown
Collaborator Author

Here i'm also not what's up - why are we not looking at the conjunct selectivity any more?

The lint failure is:

pkg/sql/opt/memo/statistics_builder.go:4122:3: this value of conjunctSelectivity is never used (SA4006)

Looked to me like the initialization was unnecessary, but if this is an actual bug we can also resolve another way.

Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB, @knz, @msbutler, @rickystewart, and @tbg)


pkg/sql/colexec/columnarizer.go line 218 at r1 (raw file):

Previously, knz (kena) wrote…

Maybe add a comment to explain we're avoiding copy because we need gcassert below.

( @yuzefovich could we use copy here? That's what the compiler is trying to tell us. )

We can replace the first for loop with the copy (it'd be nicer to put that copy before _ = newRows[len(oldRows)] line).

@rickystewart
Copy link
Copy Markdown
Collaborator Author

We can replace the first for loop with the copy (it'd be nicer to put that copy before _ = newRows[len(oldRows)] line).

Done.

@rickystewart
Copy link
Copy Markdown
Collaborator Author

Already raised this to @knz in a DM, but I'm seeing these deprecation errors in non-test code (plus the same errors in pkg/util/contextutil/context_test.go).

        pkg/util/netutil/net.go:140:52: ne.Temporary has been deprecated since Go 1.18 because it shouldn't be used: Temporary errors are not well-defined. Most "temporary" errors are timeouts, and the few exceptions are surprising. Do not use this method.  (SA1019)
    lint_test.go:1765:
        pkg/util/netutil/net.go:173:11: netError.Temporary has been deprecated since Go 1.18 because it shouldn't be used: Temporary errors are not well-defined. Most "temporary" errors are timeouts, and the few exceptions are surprising. Do not use this method.  (SA1019)

@rickystewart rickystewart requested a review from a team July 21, 2022 21:12
@rickystewart rickystewart force-pushed the upgradego118 branch 2 times, most recently from 51a4e89 to b916e66 Compare July 21, 2022 21:41
@knz
Copy link
Copy Markdown
Contributor

knz commented Jul 22, 2022

Hi Ricky
can we add a linter ignore for SA1019 on these two cases.
I have investigated this, and there's currently no better implementation in the go stdlib than the Temporary() interface, despite what the deprecation notice says. For example, an EAI_AGAIN address resolution failure in the C stdlib should be retried, but go does not offer visibility into this other than Temporary().

@rickystewart
Copy link
Copy Markdown
Collaborator Author

can we add a linter ignore for SA1019 on these two cases.

OK, done.

@rickystewart
Copy link
Copy Markdown
Collaborator Author

This is now passing CI and apparently fully ready for review. 🙏

I've added a lint ignore directive for every lint error that someone hasn't already given me a dedicated fix for.

Copy link
Copy Markdown
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: nice 💯 🔥

Reviewed 1 of 1 files at r2, 1 of 1 files at r3, 2 of 3 files at r4, 2 of 4 files at r5, 1 of 1 files at r6, 1 of 3 files at r7, 12 of 14 files at r9, 13 of 13 files at r10, 3 of 3 files at r11, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @HonoreDB, @knz, @msbutler, @rickystewart, and @tbg)

* [ ] Adjust the Pebble tests to run in new version.
* [x] Adjust version in Docker image ([source](./builder/Dockerfile)).
* [x] Adjust version in the TeamCity agent image ([setup script](./packer/teamcity-agent.sh))
* [ ] Rebuild and push the Docker image (following [Basic Process](#basic-process))
* [x] Download ALL the archives (`.tar.gz`, `.zip`) for the new Go version from https://golang.org/dl/ and mirror them in the `public-bazel-artifacts` bucket in the `Bazel artifacts` project in GCP (sub-directory `go`, next to the other Go SDK's).
* [x] Bump the version in `WORKSPACE` under `go_download_sdk`. You may need to bump [rules_go](https://github.com/bazelbuild/rules_go/releases). Also edit the filenames listed in `sdks` and update all the hashes to match what you mirrored in the step above.
* [x] Run `./dev generate bazel` to refresh `distdir_files.bzl`, then `bazel fetch @distdir//:archives` to ensure you've updated all hashes to the correct value.
* [x] Bump the version in `builder.sh` accordingly ([source](./builder.sh#L6)).
* [x] Bump the version in `go-version-check.sh` ([source](./go-version-check.sh)), unless bumping to a new patch release.
* [ ] Bump the go version in `go.mod`. You may also need to rerun `make vendor_rebuild` if vendoring has changed.
* [x] Bump the default installed version of Go in `bootstrap-debian.sh` ([source](./bootstrap/bootstrap-debian.sh)).
* [x] Replace other mentions of the older version of go (grep for `golang:<old_version>` and `go<old_version>`).
* [ ] Update the `builder.dockerImage` parameter in the TeamCity [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
* [ ] Ask the Developer Infrastructure team to deploy new TeamCity agent images according to [packer/README.md](./packer/README.md)

Release note (build change): Upgrade to golang 1.18.4
@rickystewart
Copy link
Copy Markdown
Collaborator Author

TFTR!

bors r=knz

@craig craig bot merged commit 9e44f56 into cockroachdb:master Jul 25, 2022
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 25, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants