Skip to content

storage: remove pre-22.2 code#96805

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
RaduBerinde:remove-pre-v22.2-storage
Feb 13, 2023
Merged

storage: remove pre-22.2 code#96805
craig[bot] merged 1 commit intocockroachdb:masterfrom
RaduBerinde:remove-pre-v22.2-storage

Conversation

@RaduBerinde
Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde commented Feb 8, 2023

This change removes any code that handles pre-22.2 versions. Features
that are in 22.2 (like range keys) are now always enabled.

Any new store always uses at least the 22.2 pebble format version.

Release note: None
Epic: none

Fixes: #96764

@RaduBerinde RaduBerinde requested review from a team as code owners February 8, 2023 17:23
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@RaduBerinde
Copy link
Copy Markdown
Member Author

The only material change is adding a new test case to TestMakeIngestionWriterOptions.

@RaduBerinde RaduBerinde marked this pull request as draft February 8, 2023 17:39
@RaduBerinde
Copy link
Copy Markdown
Member Author

900+ failed tests haha, obviously this needs more work. Making this a draft.

@RaduBerinde RaduBerinde force-pushed the remove-pre-v22.2-storage branch 9 times, most recently from cdabc01 to 665355c Compare February 8, 2023 23:19
@RaduBerinde RaduBerinde marked this pull request as ready for review February 8, 2023 23:19
@RaduBerinde RaduBerinde requested a review from a team February 8, 2023 23:19
@RaduBerinde RaduBerinde requested a review from a team as a code owner February 8, 2023 23:19
@RaduBerinde RaduBerinde requested a review from a team February 8, 2023 23:19
@RaduBerinde
Copy link
Copy Markdown
Member Author

Ready for review.

Copy link
Copy Markdown
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And is this safe to do because we dropped the need for 23.1 binaries to handle a transition from 22.1 due to the postponement outlined in https://groups.google.com/a/cockroachlabs.com/g/eng/c/v4hgN-rfEnk/m/VgDuolKeCQAJ?utm_medium=email&utm_source=footer?

Thanks for doing this cleanup!

:lgtm:

Reviewed 14 of 19 files at r1, 6 of 6 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @RaduBerinde)


pkg/storage/pebble.go line 1513 at r3 (raw file):

func shouldWriteLocalTimestamps(ctx context.Context, settings *cluster.Settings) bool {
	if !LocalTimestampsEnabled.Get(&settings.SV) {

let's confirm with @nvanbenschoten that there hasn't been some new thinking regarding local timestamps.


pkg/sql/gcjob_test/gc_job_test.go line 672 at r3 (raw file):

// initialization.
//
// TODO(ajwerner): Remove this test in 23.1.

can you confirm with @ajwerner


pkg/upgrade/upgrademanager/manager_external_test.go line 305 at r3 (raw file):

	// We're going to be migrating from startKey to endKey. We end up needing
	// to use real versions because the ListBetween uses the keys compiled into

why was is saying "real versions" when startMajor was 42 and our current real highest value for Major is 22?

Copy link
Copy Markdown
Member Author

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Yes, the binary min version has been bumped to v22.2, so transitioning from 22.1 is off the table.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @sumeerbhola)


pkg/upgrade/upgrademanager/manager_external_test.go line 305 at r3 (raw file):

Previously, sumeerbhola wrote…

why was is saying "real versions" when startMajor was 42 and our current real highest value for Major is 22?

Feels like the comment doesn't match the code (anymore?). In any case, I needed this test change in an earlier variant where I didn't allow older versions in SetMinVersion. But I left it because this is cleaner.

@RaduBerinde RaduBerinde force-pushed the remove-pre-v22.2-storage branch 2 times, most recently from 951c59d to 06edc6a Compare February 9, 2023 20:36
Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @RaduBerinde and @sumeerbhola)


pkg/storage/pebble.go line 1513 at r3 (raw file):

Previously, sumeerbhola wrote…

let's confirm with @nvanbenschoten that there hasn't been some new thinking regarding local timestamps.

Keeping the cluster setting but removing the version check SGTM.

This change removes any code that handles pre-22.2 versions. Features
that are in 22.2 (like range keys) are now always enabled.

Any new store always uses at least the 22.2 pebble format version.

Release note: None
Epic: none

Fixes: cockroachdb#96764
@RaduBerinde RaduBerinde force-pushed the remove-pre-v22.2-storage branch from 06edc6a to 12aec72 Compare February 13, 2023 15:28
Copy link
Copy Markdown
Member Author

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @ajwerner and @sumeerbhola)


pkg/sql/gcjob_test/gc_job_test.go line 672 at r3 (raw file):

Previously, sumeerbhola wrote…

can you confirm with @ajwerner

He is out for a couple of days, I will merge this for now and double-check with him afterward.

@RaduBerinde
Copy link
Copy Markdown
Member Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 13, 2023

Build succeeded:

@craig craig bot merged commit 47331b1 into cockroachdb:master Feb 13, 2023
@RaduBerinde RaduBerinde deleted the remove-pre-v22.2-storage branch February 13, 2023 18:02
@yuzefovich
Copy link
Copy Markdown
Member

I think this PR made some of the benchmarks fail.

On 8511dc0 ./dev bench pkg/kv/kvserver/batcheval -v --stream-output -- --test_env=COCKROACH_RANDOM_SEED=8375911443075108612 results in

goos: darwin
goarch: arm64
BenchmarkRefreshRange
    test_log_scope.go:161: test logs captured to: /Users/yuzefovich/go/src/github.com/cockroachdb/cockroach/tmp/_tmp/534984b8b0c2420fd6951b9c0f160913/logBenchmarkRefreshRange1160157368
    test_log_scope.go:79: use -show-logs to present logs inline
BenchmarkRefreshRange/linear-keys
BenchmarkRefreshRange/linear-keys/refresh_window=[0.00,0.00]
panic: pebble: range keys require at least format major version 8 (current: 5)

goroutine 56 [running]:
github.com/cockroachdb/pebble.(*DB).newIterInternal(0x14000d85900, 0x0, 0x0, 0x14000b62b08)
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/db.go:943 +0x3cc
github.com/cockroachdb/pebble.(*DB).NewIter(0x0?, 0x140001560b0?)
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/db.go:1251 +0x28
github.com/cockroachdb/cockroach/pkg/storage.newPebbleIterator({0x1077a9980, 0x14000d85900}, {0x0, {0x140001560b0, 0x5, 0x8}, {0x140001560c0, 0x8, 0x8}, 0x0, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/storage/pebble_iterator.go:90 +0x90
github.com/cockroachdb/cockroach/pkg/storage.(*Pebble).NewMVCCIterator(0x14000700180?, 0x14004e25300?, {0x0, {0x140001560b0, 0x5, 0x8}, {0x140001560c0, 0x8, 0x8}, 0x0, ...})
	github.com/cockroachdb/cockroach/pkg/storage/pebble.go:1260 +0x144
github.com/cockroachdb/cockroach/pkg/storage.NewMVCCIncrementalIterator({0x1077e9d80, 0x14000700180}, {0x2, {0x140001560b0, 0x5, 0x8}, {0x140001560c0, 0x8, 0x8}, {0x5, ...}, ...})
	github.com/cockroachdb/cockroach/pkg/storage/mvcc_incremental_iterator.go:212 +0x2a8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval.refreshRange({0x1077e9d80?, 0x14000700180?}, {{0x140001560b0, 0x5, 0x8}, {0x140001560c0, 0x8, 0x8}}, {0x134276200?, 0x28?, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval/cmd_refresh_range.go:75 +0xd8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval.RefreshRange({_, _}, {_, _}, {{0x1078340b8, 0x140072e4018}, {{0x5, 0x1, 0x0}, 0x0, ...}, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval/cmd_refresh_range.go:58 +0x3d0
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test.runRefreshRangeBenchmark.func1({0x1077b0e08, 0x1400007e078}, {0x10783e8c8?, 0x14000700180}, {0x1078340b8, 0x140072e4018}, {0x140001560b0, 0x5, 0x8}, {0x140001560c0, ...}, ...)
	github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test/pkg/kv/kvserver/batcheval/cmd_refresh_range_bench_test.go:131 +0x1e8
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test.runRefreshRangeBenchmark(0x14000773440, 0x102700e90?, {{0x5, 0x0, 0x0}, {0x5, 0x1, 0x0}, {0xf4240, 0x40, ...}})
	github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test/pkg/kv/kvserver/batcheval/cmd_refresh_range_bench_test.go:161 +0x24c
github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test.BenchmarkRefreshRange.func1.1(0x14000773440?)
	github.com/cockroachdb/cockroach/pkg/kv/kvserver/batcheval_test/pkg/kv/kvserver/batcheval/cmd_refresh_range_bench_test.go:106 +0xc8
testing.(*B).runN(0x14000773440, 0x1)
	GOROOT/src/testing/benchmark.go:193 +0x12c
testing.(*B).run1.func1()
	GOROOT/src/testing/benchmark.go:233 +0x50
created by testing.(*B).run1
	GOROOT/src/testing/benchmark.go:226 +0x90
I230213 23:50:01.899850 1 (gostd) testmain.go:176  [T1] 1  Test //pkg/kv/kvserver/batcheval:batcheval_test exited with error code 2

@RaduBerinde
Copy link
Copy Markdown
Member Author

RaduBerinde commented Feb 14, 2023

Thanks, yes, #97066 will hopefully fix that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: remove V22_2 gates and related logic/migrations/etc

5 participants