Skip to content

sql,descs: add & adopt descriptor_validation session var#90488

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
postamar:fix-50651
Oct 31, 2022
Merged

sql,descs: add & adopt descriptor_validation session var#90488
craig[bot] merged 1 commit intocockroachdb:masterfrom
postamar:fix-50651

Conversation

@postamar
Copy link
Copy Markdown

This commit removes the sql.catalog.descs.validate_on_write.enabled cluster setting and replaces it with the descriptor_validation session variable. The default value is 'on' which indicates that catalog descriptors are to be validated when both read from- and written to the system.descriptor table. Other possible values are 'off' which disables validation entirely and 'read_only' which disables it for writes.

Informs #50651.

Release note (sql change): added a new 'descriptor_validation' session variable which can be set to 'read_only' or 'off' to disable descriptor validation, which may be useful when mitigating or recovering from catalog corruption.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@postamar postamar force-pushed the fix-50651 branch 4 times, most recently from 08d1469 to a15b492 Compare October 27, 2022 00:54
@postamar postamar marked this pull request as ready for review October 27, 2022 12:21
@postamar postamar requested a review from a team as a code owner October 27, 2022 12:21
@postamar postamar requested a review from a team October 27, 2022 12:21
@postamar postamar requested review from a team as code owners October 27, 2022 12:21
@postamar postamar requested review from ajwerner and stevendanna and removed request for a team and stevendanna October 27, 2022 12:21
@postamar
Copy link
Copy Markdown
Author

@ajwerner this is ready for review. This commit only covers part of the work required for #50651 but I'd like to merge it ahead of the rest because it has enough merit of its own and offers a strict improvement over the use of a cluster setting for controlling descriptor validation behaviour.

For the actual DROP story I'm not quite sure what to do yet and I'll want to explore. Contrary to initial impressions perhaps the declarative schema changer will prove more robust after all. This would involve making scdecomp robust in the face of descriptor corruption. If this turns out to be possible, then I have a feeling this is going to be preferable, because the output of EXPLAIN (DDL) DROP ... might then be enough for a power user/support specialist to determine whether it's safe to go ahead or not.

Copy link
Copy Markdown
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: this is a good change. Just the nit on narrowing the interface scope for sessiondata. I could see building a little adapter which has the methods you really need. It's a bit of ceremony. Maybe like:

package descs

type SessionData interface {
    TemporarySchemaProvider
    ValidationMode() sessiondatapb.ValidationMode
}
package catsessiondata

type SessionData sessiondata.Stack

func MakeSessionData(sds *sessiondata.Stack) *SesssionData {   return (*SessionData)(sds) }

func (sd *SessionData) toStack() *sessiondata.Stack { return (*sessiondata.Stack)(sd) }

func (sd *SessionData) ValidationMode() sessiondatapb.ValidationMode { 
    return sd.toStack().Top().ValidationMode()
}

Reviewed 35 of 45 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @postamar)


pkg/sql/catalog/descs/collection.go line 51 at r1 (raw file):

	systemDatabase *catkv.SystemDatabaseCache,
	virtualSchemas catalog.VirtualSchemas,
	sds *sessiondata.Stack,

nit: how would you feel about narrowing the scope of the interface you need here to just the relevant parts of the sessiondata.Stack? Time has shown that narrowing the dependencies prevents sprawl and increases clarity.

@postamar
Copy link
Copy Markdown
Author

postamar commented Oct 27, 2022

Thanks for the suggestion. I hesitated to do that, worrying it might be overkill, but now I'll do it. I might have to implement that interface both on SessionData and on Stack, because AFAIK the stack has a different lifecycle than the collection: the collection gets reset every txn but the stack may change within the txn. That doesn't bother me, though.

@postamar postamar requested a review from a team as a code owner October 28, 2022 15:59
@postamar postamar force-pushed the fix-50651 branch 2 times, most recently from b0eb698 to 3ab5918 Compare October 28, 2022 16:04
@postamar
Copy link
Copy Markdown
Author

Assuming that CI is happy, this is ready for another look. I think your suggested changes were worth it.

This commit removes the `sql.catalog.descs.validate_on_write.enabled`
cluster setting and replaces it with the `descriptor_validation` session
variable. The default value is 'on' which indicates that catalog
descriptors are to be validated when both read from- and written to
the system.descriptor table. Other possible values are 'off' which
disables validation entirely and 'read_only' which disables it for
writes.

This commit therefore plumbs the session data into the descs.Collection
object and for this purpose defines a new DescriptorSessionDataProvider
interface, which is implemented in a new catsessiondata package.

Informs cockroachdb#50651.

Release note (sql change): added a new 'descriptor_validation' session
variable which can be set to 'read_only' or 'off' to disable descriptor
validation, which may be useful when mitigating or recovering from
catalog corruption.
Copy link
Copy Markdown
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm_strong:

Reviewed 42 of 48 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

@postamar
Copy link
Copy Markdown
Author

Thanks for the reviews!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Oct 31, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants