Skip to content

sql: add virtual indexes to crdb_internal.cluster_locks virtual table #79623

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
AlexTalks:cluster_locks_index
May 14, 2022
Merged

sql: add virtual indexes to crdb_internal.cluster_locks virtual table #79623
craig[bot] merged 1 commit intocockroachdb:masterfrom
AlexTalks:cluster_locks_index

Conversation

@AlexTalks
Copy link
Copy Markdown
Contributor

@AlexTalks AlexTalks commented Apr 7, 2022

This change adds virtual indexes on the table_id, database_name, and
table_name columns of the crdb_internal.cluster_locks virtual table,
so that when queried, the kv.Batches with QueryLocksRequests can be
constrained to query only specific tables or databases. This allows the
requests to be much more limited, rather than needing to request all of
the ranges that comprise the key spans of all tables accessible by the
user.

Release note (sql change): Improved query performance for
crdb_internal.cluster_locks when issued with constraints in the WHERE
clause on table_id, database_name, or table_name columns.

Depends on #77876, #80422

@AlexTalks AlexTalks requested a review from a team as a code owner April 7, 2022 23:04
@AlexTalks AlexTalks requested review from a team April 7, 2022 23:04
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @AlexTalks and @rafiss)


pkg/sql/crdb_internal.go, line 5837 at r3 (raw file):

    contended           BOOL,
    duration            INTERVAL,
    INDEX(table_id),

Do we have tests that exercise each of these indexes?


pkg/sql/crdb_internal.go, line 5844 at r3 (raw file):

	indexes: []virtualIndex{
		{
			populate: func(

Could you give each of these objects that somehow indicate which INDEX they're implementing?


pkg/sql/crdb_internal.go, line 6023 at r3 (raw file):

				IncludeUncontended: true,
			}
			if filters.contended != nil && *filters.contended {

If !*filters.contended, don't we want to include only uncontended locks?

@AlexTalks AlexTalks force-pushed the cluster_locks_index branch 2 times, most recently from 4504d7f to 6b66286 Compare April 25, 2022 20:23
Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @rafiss)


pkg/sql/crdb_internal.go line 5837 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Do we have tests that exercise each of these indexes?

OK - Added these! I'm not sure if I should also add a small test that double checks the query plan to make sure it's using the index? If so let me know!


pkg/sql/crdb_internal.go line 5844 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Could you give each of these objects that somehow indicate which INDEX they're implementing?

Not sure I understand - you mean comments? Or you mean rename unwrappedConstraint to, say, tableNameConstraint for example?


pkg/sql/crdb_internal.go line 6023 at r3 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

If !*filters.contended, don't we want to include only uncontended locks?

Yes - filtering this out, however, is already handled by the virtual schema table - only the rows that match the index constraint will be returned anyway, so we can instead focus here on limiting our request.

@AlexTalks AlexTalks force-pushed the cluster_locks_index branch from bdf1923 to e46eafe Compare April 25, 2022 23:30
@AlexTalks AlexTalks force-pushed the cluster_locks_index branch from e46eafe to 12d37b0 Compare April 26, 2022 01:23
@blathers-crl blathers-crl bot requested a review from otan April 26, 2022 01:23
@AlexTalks AlexTalks force-pushed the cluster_locks_index branch from 12d37b0 to a5e3892 Compare April 26, 2022 01:50
Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten, @otan, and @rafiss)


pkg/sql/logictest/testdata/logic_test/cluster_locks line 149 at r5 (raw file):

Previously, otan (Oliver Tan) wrote…

can you add a test for WHERE table_id = ... and WHERE table_name = ...?

to get the tableid, you can probably do WHERE table_id = 'table_name'::regclass::oid::int.

OK - had a few with table_name = ... previously, but I just added one with table_id.

Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dismissed @otan from a discussion.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten, @otan, and @rafiss)

Copy link
Copy Markdown
Contributor Author

@AlexTalks AlexTalks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten and @rafiss)


pkg/sql/crdb_internal.go line 5844 at r3 (raw file):

Previously, AlexTalks (Alex Sarkesian) wrote…

Not sure I understand - you mean comments? Or you mean rename unwrappedConstraint to, say, tableNameConstraint for example?

Done. I refactored these a bit to use a generator, with the index-specific bits of logic passed in so it should now be pretty clear what these are.

This change adds virtual indexes on the `table_id`, `database_name`, and
`table_name` columns of the `crdb_internal.cluster_locks` virtual table,
so that when queried, the `kv.Batch`es with `QueryLocksRequest`s can be
constrained to query only specific tables or databases. This allows the
requests to be much more limited, rather than needing to request all of
the ranges that comprise the key spans of all tables accessible by the
user.

Release note (sql change): Improved query performance for
`crdb_internal.cluster_locks` when issued with constraints in the WHERE
clause on `table_id`, `database_name`, or `table_name` columns.
@AlexTalks AlexTalks force-pushed the cluster_locks_index branch from a5e3892 to f8a75ac Compare May 13, 2022 22:38
@AlexTalks
Copy link
Copy Markdown
Contributor Author

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented May 14, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants