Skip to content

sql: fix bug with primary key swap not setting encoding and columns#45093

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:pk-change-swap-fix
Feb 20, 2020
Merged

sql: fix bug with primary key swap not setting encoding and columns#45093
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:pk-change-swap-fix

Conversation

@rohany
Copy link
Copy Markdown
Contributor

@rohany rohany commented Feb 13, 2020

This PR fixes a bug where the primary key swap operation
during the primary key change process would not set the
encoding for the old primary index to be PrimaryIndexEncoding,
and would not include the columns that the old index stores.
This could cause a variety of potential errors, such as
errors during updates and inserts while some nodes have the
old primary index in the DELETE_AND_WRITE_ONLY state
and others still have the old index as the primary key.

Release note: None

@rohany rohany requested a review from solongordon February 13, 2020 19:30
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rohany rohany force-pushed the pk-change-swap-fix branch from 6bc44f4 to d52ac9e Compare February 13, 2020 19:32
Copy link
Copy Markdown
Contributor

@andy-kimball andy-kimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rohany and @solongordon)


pkg/sql/sqlbase/structured.go, line 2991 at r1 (raw file):

			// Actually write the new primary index into the descriptor, and remove it from the indexes list.
			// Additionally, schedule the old primary index for deletion. Note that if needed, a copy of the primary index

How come we'd create a copy of the primary index that doesn't store any columns, and then create another copy below that does store all columns? Why wouldn't the primary index always store all columns to begin with?

Copy link
Copy Markdown
Contributor Author

@rohany rohany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball and @solongordon)


pkg/sql/sqlbase/structured.go, line 2991 at r1 (raw file):

Previously, andy-kimball (Andy Kimball) wrote…

How come we'd create a copy of the primary index that doesn't store any columns, and then create another copy below that does store all columns? Why wouldn't the primary index always store all columns to begin with?

The code implicitly assumes everywhere that primary indexes store all the columns in the table -- the StoreColumn* fields are only populated for secondary indexes. When we demote the primary index here we have to now populate these fields.

Where do you see two copies of the index being created? I might be misunderstanding your comment.

Copy link
Copy Markdown
Contributor

@andy-kimball andy-kimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rohany and @solongordon)


pkg/sql/sqlbase/structured.go, line 2991 at r1 (raw file):

Previously, rohany (Rohan Yadav) wrote…

The code implicitly assumes everywhere that primary indexes store all the columns in the table -- the StoreColumn* fields are only populated for secondary indexes. When we demote the primary index here we have to now populate these fields.

Where do you see two copies of the index being created? I might be misunderstanding your comment.

I don't know details - I was just reacting to the juxtaposition of these two comments:

// Additionally, schedule the old primary index for deletion. Note that if needed, *a copy of the primary index*
// *that doesn't store any columns* was created at the beginning of the primary key change operation.

and

// the schema change stages are correct. Additionally, we want to denote that
// *this index stores all the columns in the table.*

Just seemed strange to create a copy without stored columns, and then a copy with them, if that's indeed what's happening.

Copy link
Copy Markdown
Contributor Author

@rohany rohany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball and @solongordon)


pkg/sql/sqlbase/structured.go, line 2991 at r1 (raw file):

Previously, andy-kimball (Andy Kimball) wrote…

I don't know details - I was just reacting to the juxtaposition of these two comments:

// Additionally, schedule the old primary index for deletion. Note that if needed, *a copy of the primary index*
// *that doesn't store any columns* was created at the beginning of the primary key change operation.

and

// the schema change stages are correct. Additionally, we want to denote that
// *this index stores all the columns in the table.*

Just seemed strange to create a copy without stored columns, and then a copy with them, if that's indeed what's happening.

Ah I see -- I think I can word this better. Whats happening is this:

When you alter the primary key, by default we create a new secondary index that indexes all of the same columns as the primary key, but doesn't store any columns. This is so that user's workloads that depend on the primary index don't suddenly start to regress in performance. That index is separate from the primary index that is being deleted right here and moved into the mutations queue.

@rohany rohany force-pushed the pk-change-swap-fix branch from d52ac9e to 0aaf4c2 Compare February 19, 2020 18:24
Copy link
Copy Markdown
Contributor

@solongordon solongordon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good aside from one confusing comment.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball, @rohany, and @solongordon)


pkg/sql/sqlbase/structured.go, line 2997 at r2 (raw file):

			// We need to make the old primary key denoted with the PrimaryIndexEncoding
			// so that operations on it that rely on its encoding succeed while it is in
			// the schema change stages are correct. Additionally, we want to denote that

This sentence is kind of breaking my brain when I try to parse it. How about something like this: "Update the old primary index's descriptor to denote that it uses the PrimaryIndexEncoding and stores all columns. This ensures that it will be properly decoded if it is accessed after it is no longer the primary key but before it is dropped entirely." That's probably not quite right but you get the idea.

@rohany rohany force-pushed the pk-change-swap-fix branch from 0aaf4c2 to b0a9190 Compare February 20, 2020 16:41
@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Feb 20, 2020

bors r=solongordon

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 20, 2020

Merge conflict (retrying...)

@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Feb 20, 2020

bors r-

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 20, 2020

Canceled

This PR fixes a bug where the primary key swap operation
during the primary key change process would not set the
encoding for the old primary index to be PrimaryIndexEncoding,
and would not include the columns that the old index stores.
This could cause a variety of potential errors, such as
errors during updates and inserts while some nodes have the
old primary index in the `DELETE_AND_WRITE_ONLY` state
and others still have the old index as the primary key.

Release note: None
@rohany rohany force-pushed the pk-change-swap-fix branch from b0a9190 to c4e7eda Compare February 20, 2020 19:02
@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Feb 20, 2020

bors r=solongordon

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 20, 2020

Merge conflict (retrying...)

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Feb 20, 2020

Build succeeded

@craig craig bot merged commit ff6641d into cockroachdb:master Feb 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants