Skip to content

sql: add sql grammar for inspect command#151067

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
bghal:inspect-command
Sep 4, 2025
Merged

sql: add sql grammar for inspect command#151067
craig[bot] merged 1 commit intocockroachdb:masterfrom
bghal:inspect-command

Conversation

@bghal
Copy link
Copy Markdown
Contributor

@bghal bghal commented Jul 30, 2025

The INSPECT commands are being added to support data consistency
validation.
These new statements require new SQL grammar.
The grammar is added in this change and the implementations will be
added in future PRs.

Epic: CRDB-30356
Part of: #148272

Release note (sql change): Introduces the INSPECT TABLE and INSPECT DATABASE statements that are unimplemented. The new
enable_inspect_command cluster setting feature flag configures access
to the new features as they're implemented.

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Jul 30, 2025

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

It looks like your PR touches SQL parser code but doesn't add or edit parser tests. Please make sure you add or edit parser tests if you edit the parser.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@bghal bghal force-pushed the inspect-command branch 2 times, most recently from 271289d to e18d598 Compare July 30, 2025 20:16
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Jul 30, 2025

It looks like your PR touches SQL parser code but doesn't add or edit parser tests. Please make sure you add or edit parser tests if you edit the parser.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@bghal bghal force-pushed the inspect-command branch 4 times, most recently from 44092be to 3e52bbc Compare July 30, 2025 20:52
Copy link
Copy Markdown
Contributor

@spilchen spilchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good. I had a few minor comments.

Reviewed 18 of 23 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained


pkg/sql/inspect_node.go line 38 at r1 (raw file):

	case tree.InspectTable:
		if !params.p.extendedEvalCtx.SessionData().EnableInspectCommand {
			return errors.New("INSPECT TABLE is not enabled by enable_inspect_command")

We have a package for unimlemented errors (see unimplemented.Newf). We should use that here.


pkg/sql/parser/testdata/inspect line 0 at r1 (raw file):
Can we have a testcase where we accept multiple index names? And can we have a test that uses multi-part names for the table?


docs/generated/sql/bnf/inspect_database_stmt.bnf line 2 at r1 (raw file):

inspect_database_stmt ::=
	'INSPECT' 'DATABASE' database_name opt_as_of_clause opt_inspect_options_clause

Since the command isn't ready for prime time yet, we should not have it documented. You can add hidden SQL grammar with the /* SKIP DOC */ comment. Once added, these updates to docs/ should be undone.


pkg/sql/sem/tree/inspect.go line 15 at r1 (raw file):

const (
	// InspectTable describes the INSPECT operation INSPECT TABLE.
	InspectTable = iota

Should we use the InspectType type for these const?


pkg/sql/sessiondatapb/local_only_session_data.proto line 730 at r1 (raw file):

  // mutations), regardless of the table being multi-region.
  bool parallelize_multi_key_lookup_joins_only_on_mr_mutations = 182 [(gogoproto.customname) = "ParallelizeMultiKeyLookupJoinsOnlyOnMRMutations"];
  // EnableInspectCommand controls if a job is initiated when the INSPECT

nit: Is this comment accurate? It's just enables the use of the INSPECT command. It doesn't control if a job is used or not.


pkg/sql/parser/sql.y line 7863 at r1 (raw file):

// %Category: Experimental
// %Text:
// EXPERIMENTAL INSPECT DATABASE <database>

nit: as discussed, lets drop EXPERIMENTAL here


pkg/sql/parser/sql.y line 7873 at r1 (raw file):

  INSPECT DATABASE database_name opt_as_of_clause opt_inspect_options_clause
  {
    $$.val = &tree.Inspect{Typ: tree.InspectDatabase, Database: tree.Name($3), AsOf: $4.asOfClause()}

Should we use the options field? i.e. $5.inspectOptions()

@bghal bghal force-pushed the inspect-command branch 2 times, most recently from 416479f to f3092ae Compare August 8, 2025 23:29
Copy link
Copy Markdown
Contributor Author

@bghal bghal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @spilchen)


pkg/sql/inspect_node.go line 38 at r1 (raw file):

Previously, spilchen wrote…

We have a package for unimlemented errors (see unimplemented.Newf). We should use that here.

Done.


pkg/sql/parser/sql.y line 7873 at r1 (raw file):

Previously, spilchen wrote…

Should we use the options field? i.e. $5.inspectOptions()

Done.


pkg/sql/sem/tree/inspect.go line 15 at r1 (raw file):

Previously, spilchen wrote…

Should we use the InspectType type for these const?

Done.


docs/generated/sql/bnf/inspect_database_stmt.bnf line 2 at r1 (raw file):

Previously, spilchen wrote…

Since the command isn't ready for prime time yet, we should not have it documented. You can add hidden SQL grammar with the /* SKIP DOC */ comment. Once added, these updates to docs/ should be undone.

Done.

Copy link
Copy Markdown
Contributor Author

@bghal bghal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @spilchen)


pkg/sql/parser/testdata/inspect line at r1 (raw file):

Previously, spilchen wrote…

Can we have a testcase where we accept multiple index names? And can we have a test that uses multi-part names for the table?

Done.

@bghal bghal marked this pull request as ready for review August 12, 2025 14:30
@bghal bghal requested review from a team and spilchen August 12, 2025 14:30
@bghal bghal force-pushed the inspect-command branch 2 times, most recently from 47056be to 76b163b Compare August 25, 2025 22:42
Copy link
Copy Markdown
Contributor

@spilchen spilchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spilchen reviewed 2 of 23 files at r1, 1 of 3 files at r2, 5 of 18 files at r3, 11 of 11 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @bghal)


pkg/sql/inspect_node.go line 38 at r1 (raw file):

Previously, bghal wrote…

Done.

nit: I would suggest slightly different wording:

"INSPECT TABLE requires the enable_inspect_command setting to be enabled"
"INSPECT DATABASE requires the enable_inspect_command setting to be enabled"

pkg/sql/parser/testdata/inspect line at r1 (raw file):

Previously, bghal wrote…

Done.

Hmm, I still don't see these extra cases. I was thinking of:

  • INSPECT TABLE db.schema.t1
  • INSPECT TABLE schema.t1 WITH OPTIONS INDEX ( 'i1', 'i2' );

pkg/sql/sem/tree/stmt.go line 1383 at r4 (raw file):

// StatementReturnType implements the Statement interface.
func (*Inspect) StatementReturnType() StatementReturnType { return Rows }

I don't think we should use Rows. I feel like Ack might be better. INSPECT isn't returning the rows with inconsistencies. If validation passes, nothing is returned to the client. The command will just be successful. If there is an inconsistency detected, the command will fail. The user will need to look at the errors table to see what the inconsistency is.

Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'll defer to Matt for the final review, just leaving a couple small comments. (not marking as blocking since i'm going OOO, but please address them)

overall lgtm!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @bghal and @spilchen)


pkg/sql/inspect_node.go line 27 at r5 (raw file):

// Privileges: superuser.
func (p *planner) Inspect(ctx context.Context, n *tree.Inspect) (planNode, error) {
	if err := p.CheckPrivilege(ctx, syntheticprivilege.GlobalPrivilegeObject, privilege.ALL); err != nil {

we should be using the INSPECT privilege here (and update the function comment to mention that too)


pkg/sql/parser/sql.y line 6897 at r5 (raw file):

| import_stmt    // EXTEND WITH HELP: IMPORT
| insert_stmt    // EXTEND WITH HELP: INSERT
| inspect_stmt   { /* SKIP DOC */ }

nit: let's not skip the docs for this. it's going to be a "preview" feature, but those still should get documented: https://www.cockroachlabs.com/docs/stable/cockroachdb-feature-availability


pkg/sql/parser/sql.y line 7826 at r5 (raw file):

// %Help: INSPECT - run checks against databases or tables
// %Category: Experimental

nit: let's remove "experimental" from here. that's an older term that was more restrictive than "preview"


pkg/sql/parser/sql.y line 7834 at r5 (raw file):

inspect_stmt:
  inspect_table_stmt { /* SKIP DOC */ }
| inspect_database_stmt { /* SKIP DOC */ }

nit: let's not skip the docs


pkg/sql/parser/sql.y line 7838 at r5 (raw file):

// %Help: INSPECT TABLE - run inspect checks on a table
// %Category: Experimental

nit: remove "experimental"

Copy link
Copy Markdown
Contributor

@spilchen spilchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spilchen reviewed 11 of 14 files at r5, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @bghal and @rafiss)


pkg/sql/parser/sql.y line 6897 at r5 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

nit: let's not skip the docs for this. it's going to be a "preview" feature, but those still should get documented: https://www.cockroachlabs.com/docs/stable/cockroachdb-feature-availability

Sorry, this is my bad. I had suggested to skip the docs. The thinking was we would add the docs later. But it'll save a step if we add the docs now.


pkg/sql/parser/testdata/inspect line at r1 (raw file):

Previously, spilchen wrote…

Hmm, I still don't see these extra cases. I was thinking of:

  • INSPECT TABLE db.schema.t1
  • INSPECT TABLE schema.t1 WITH OPTIONS INDEX ( 'i1', 'i2' );

I see multiple indexes now. That's good. But I think we are still missing the 3 part name.

Copy link
Copy Markdown
Contributor Author

@bghal bghal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bghal dismissed @spilchen from a discussion.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss and @spilchen)


pkg/sql/inspect_node.go line 27 at r5 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

we should be using the INSPECT privilege here (and update the function comment to mention that too)

Done.


pkg/sql/sem/tree/stmt.go line 1383 at r4 (raw file):

Previously, spilchen wrote…

I don't think we should use Rows. I feel like Ack might be better. INSPECT isn't returning the rows with inconsistencies. If validation passes, nothing is returned to the client. The command will just be successful. If there is an inconsistency detected, the command will fail. The user will need to look at the errors table to see what the inconsistency is.

Done.


pkg/sql/parser/testdata/inspect line at r1 (raw file):

Previously, spilchen wrote…

Hmm, I still don't see these extra cases. I was thinking of:

  • INSPECT TABLE db.schema.t1
  • INSPECT TABLE schema.t1 WITH OPTIONS INDEX ( 'i1', 'i2' );

Think I added and deleted those 🙃.

The `INSPECT` commands are being added to support data consistency
validation.
These new statements require new SQL grammar.
The grammar is added in this change and the implementations will be
added in future PRs.

Epic: CRDB-30356
Part of: cockroachdb#148272

Release note (sql change): Introduces the `INSPECT TABLE` and `INSPECT
DATABASE` statements that are unimplemented. The new
`enable_inspect_command` cluster setting feature flag configures access
to the new features as they're implemented.
@bghal bghal requested a review from a team as a code owner September 3, 2025 22:25
Copy link
Copy Markdown
Contributor

@spilchen spilchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Thanks for doing the revisions.

@spilchen reviewed 1 of 14 files at r5, 4 of 5 files at r6, 10 of 10 files at r7, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rafiss)

@bghal
Copy link
Copy Markdown
Contributor Author

bghal commented Sep 4, 2025

bors r+

craig bot pushed a commit that referenced this pull request Sep 4, 2025
151067: sql: add sql grammar for inspect command r=bghal a=bghal

The `INSPECT` commands are being added to support data consistency
validation.
These new statements require new SQL grammar.
The grammar is added in this change and the implementations will be
added in future PRs.

Epic: CRDB-30356
Part of: #148272

Release note (sql change): Introduces the `INSPECT TABLE` and `INSPECT
DATABASE` statements that are unimplemented. The new
`enable_inspect_command` cluster setting feature flag configures access
to the new features as they're implemented.


151811: rfcs: tiniest spelling fix r=bghal a=bghal

TSIA

Epic: none

Release note: None


Co-authored-by: Brendan Gerrity <brendan.gerrity@cockroachlabs.com>
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 4, 2025

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 4, 2025

@craig craig bot merged commit 2b821ef into cockroachdb:master Sep 4, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants