Skip to content

sql: separate "internal key" and "display name" for system privileges#111653

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
knz:20231003-priv-ids-vs-names
Oct 21, 2023
Merged

sql: separate "internal key" and "display name" for system privileges#111653
craig[bot] merged 1 commit intocockroachdb:masterfrom
knz:20231003-priv-ids-vs-names

Conversation

@knz
Copy link
Copy Markdown
Contributor

@knz knz commented Oct 3, 2023

In a way akin to 56e07c1, this change introduces a distinction between the name of a system privilege, which is the one facing end-users; and the internal key of the privilege, which is stored in system.privileges.

This distinction exists to allow us to rename the user-facing privileges towards better UX and clearer docs without having to rewrite the contents of system.privileges.

The internal key is meant to remain immutable after a privilege is added in the source code. For privileges that will be added in the future, we can now even envision storing numbers or UUIDs instead in system.privileges.

For backward-compatibility, the internal key for a system privilege is also recognized as input in SQL syntax.

(An argument could be made that we would be better served by system.privilege containing an array of integers with the internal IDs, but this change aims to avoid any migration whatsoever, and currently the system table already contains strings.)

Informs #110927.
Epic: CRDB-29380

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

return "CREATE"
case DROP:
return "DROP"
case DEPRECATEDGRANT:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only one that is different? I think it can be removed entirely now that it was migrated a few versions ago.

Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this direction seems ok to go in, thanks

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @knz)


pkg/sql/crdb_internal.go line 8580 at r1 (raw file):

CREATE VIEW crdb_internal.kv_system_privileges AS
SELECT username, path,
       crdb_internal.privilege_name(privileges),

nit: use AS to give these well-defined column names


pkg/sql/privilege/kind.go line 74 at r1 (raw file):

// KindKey is the value stored in system tables, etc, that represent
// the privilege internally. It is not visible to end-users.
type KindInternalKey string

why did we get rid of the stringer here? it seems that func (Kind) String() is no longer defined, but it was still used above. also, it seems less error-prone since with this change it's easy for someone to forget to update this switch statement


pkg/sql/privilege/privilege.go line 303 at r1 (raw file):

		case OriginFromUserInput:
			var ok bool
			k, ok = ByDisplayName[KindDisplayName(strings.ToUpper(s))]

don't we need to also keep allowing the user to specify the internal key?

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch from 987df64 to 3fa8ec0 Compare October 9, 2023 02:22
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm picking up this from Raphael to push it over the finish line.

Reviewed 26 of 26 files at r1, 11 of 11 files at r2, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @knz, and @rafiss)


pkg/sql/crdb_internal.go line 8580 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

nit: use AS to give these well-defined column names

Done.


pkg/sql/privilege/kind.go line 74 at r1 (raw file):
My guess is that the rationale is something like this: the removal of automatically generated String() implementation allows us to provide custom DisplayName() to make user-visible changes as we please. The fact that String() is no longer implemented pushes engineers to choose either InternalKey or DisplayName.

However, it does make it so that our "lintonbuild" config fails to compile because in many (?) places we use privilege.Kind with %s format directive which is no longer valid. @knz can you share your perspective on this before I start updating all those places? (I'm assuming in vast majority we'll be using DisplayName, right?

it seems less error-prone since with this change it's easy for someone to forget to update this switch statement

I don't think it's a concern because in init() below we iterate over all kinds, and for all non-deprecated ones we call InternalKey; thus, if someone forgets to add another case to the switch, the build will be red.


pkg/sql/privilege/kind.go line 84 at r1 (raw file):

Previously, ecwall (Evan Wall) wrote…

Is this the only one that is different? I think it can be removed entirely now that it was migrated a few versions ago.

Ack, I removed it.


pkg/sql/privilege/privilege.go line 303 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

don't we need to also keep allowing the user to specify the internal key?

Yes, I believe we need to, done.

Copy link
Copy Markdown
Contributor Author

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @rafiss)


pkg/sql/privilege/kind.go line 74 at r1 (raw file):

The fact that String() is no longer implemented pushes engineers to choose either InternalKey or DisplayName.

correct.

I'm assuming in vast majority we'll be using DisplayName, right?

correct, but we'd need to be careful for those cases that generate SQL syntax.


pkg/sql/privilege/privilege.go line 303 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

Yes, I believe we need to, done.

Can you double check? I intended the ByDisplayName map to be already initialized (via init function) to contain both names and internal keys.

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch 3 times, most recently from b7104a7 to 34c68c7 Compare October 9, 2023 20:40
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the CI should be green, so this is RFAL, @ecwall @rafiss

Reviewed 2 of 3 files at r3, 1 of 1 files at r4, 3 of 3 files at r5, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @knz, and @rafiss)


pkg/sql/privilege/kind.go line 74 at r1 (raw file):

Previously, knz (Raphael 'kena' Poss) wrote…

The fact that String() is no longer implemented pushes engineers to choose either InternalKey or DisplayName.

correct.

I'm assuming in vast majority we'll be using DisplayName, right?

correct, but we'd need to be careful for those cases that generate SQL syntax.

Turns out that there were just a handful of places like this, and in all of them I think DisplayName() is the right choice.


pkg/sql/privilege/privilege.go line 303 at r1 (raw file):

Previously, knz (Raphael 'kena' Poss) wrote…

Can you double check? I intended the ByDisplayName map to be already initialized (via init function) to contain both names and internal keys.

Oh, right, I missed that, reverted my change.


pkg/sql/privilege/privilege.go line 136 at r4 (raw file):

	for i, p := range pl {
		if i > 0 {
			s.SafeString(", ")

Want to bring attention to this change - previously we had privilege.List.String() which joined strings with ", " separator, we removed that implementation, so in some logic tests we started seeing a different error message where multiple privileges were joined just by a comma. Adding a space here should fix those tests, but it also seems like a nice change on its own. Thoughts?

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch 2 times, most recently from 89863ee to 01bf9a4 Compare October 9, 2023 21:27
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r6, 4 of 4 files at r7, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @knz, and @rafiss)


pkg/sql/logictest/testdata/logic_test/grant_database line 12 at r7 (raw file):

a              root     ALL             true

statement error user root must have exactly \[ALL\] privileges on database "a"

Here is another minor difference - we now use privilege.List.SortedDisplayNames in a few places which returns []string, and when that gets stringified, we see square brackets around the names. Seems reasonable.

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch from 01bf9a4 to 0674b5b Compare October 9, 2023 22:54
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 3 of 3 files at r8, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @knz, and @rafiss)


pkg/sql/crdb_internal.go line 8609 at r8 (raw file):

       crdb_internal.privilege_name(privileges) AS privileges,
       crdb_internal.privilege_name(grant_options) AS grant_options,
       user_id

Hm, there is a difficulty here: user_id column was added to system.privileges during 23.1 cycle, so in mixed 22.2-23.1+ cluster this view might result in an error if it happens to be queried on the older node. I'm not quite sure what to do here since I don't think that we have infrastructure to gate the schema of this virtual view based on the cluster version.

Do we actually need to care about this 22.2-23.1+ cluster state? I don't think we're implementing version skipping from 22.2 to 23.2, so it seems that we actually don't care about this. This would mean that we add a skip for local-mixed-22_2-23_1 logic test config to a couple of files.

The view is currently only used to power SHOW GRANTS which doesn't actually use user_id column, so in theory we could adjust the schema to omit this column. This view was added recently in #104565 and was backported to 22.2 and 23.1.

@ecwall thoughts on this?

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch 2 times, most recently from e9a479e to b3b2c63 Compare October 10, 2023 00:34
@yuzefovich yuzefovich marked this pull request as ready for review October 10, 2023 01:20
@yuzefovich yuzefovich requested review from a team as code owners October 10, 2023 01:20
@yuzefovich yuzefovich requested a review from a team October 10, 2023 01:20
@yuzefovich yuzefovich requested a review from a team as a code owner October 10, 2023 01:20
@yuzefovich yuzefovich requested a review from a team October 10, 2023 01:20
@yuzefovich yuzefovich requested a review from a team as a code owner October 10, 2023 01:20
@yuzefovich yuzefovich requested review from a team and msbutler and removed request for a team October 10, 2023 01:20
@yuzefovich yuzefovich requested review from ericharmeling and removed request for a team October 10, 2023 01:20
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for all the pushes, this is RFAL, @ecwall @rafiss.

Reviewed 2 of 2 files at r9, 3 of 3 files at r10, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @ericharmeling, @knz, @msbutler, and @rafiss)


pkg/sql/crdb_internal.go line 8609 at r8 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

Hm, there is a difficulty here: user_id column was added to system.privileges during 23.1 cycle, so in mixed 22.2-23.1+ cluster this view might result in an error if it happens to be queried on the older node. I'm not quite sure what to do here since I don't think that we have infrastructure to gate the schema of this virtual view based on the cluster version.

Do we actually need to care about this 22.2-23.1+ cluster state? I don't think we're implementing version skipping from 22.2 to 23.2, so it seems that we actually don't care about this. This would mean that we add a skip for local-mixed-22_2-23_1 logic test config to a couple of files.

The view is currently only used to power SHOW GRANTS which doesn't actually use user_id column, so in theory we could adjust the schema to omit this column. This view was added recently in #104565 and was backported to 22.2 and 23.1.

@ecwall thoughts on this?

I added a skip for local-mixed-22_2-23_1 in two places. This means that SHOW SYSTEM GRANTS might not work in mixed version 22.2-23.2 cluster when issued against the older node (apparently, we'll have experimental support for this). The alternative is to change the schema to remove user_id column, but this would make the virtual table schema differ from the system table, so I chose the skip option.

@knz
Copy link
Copy Markdown
Contributor Author

knz commented Oct 10, 2023

Do we actually need to care about this 22.2-23.1+ cluster state? I don't think we're implementing version skipping from 22.2 to 23.2

Indeed, we have chosen not to implement version skipping from 22.2 to 23.2 so I think we are safe.

@knz knz requested a review from rafiss October 10, 2023 13:40
@yuzefovich yuzefovich added the backport-23.2.x PAST MAINTENANCE SUPPORT: 23.2 patch releases via ER request only label Oct 10, 2023
@yuzefovich yuzefovich requested a review from ecwall October 11, 2023 05:17
@yuzefovich
Copy link
Copy Markdown
Member

Friendly ping @rafiss @ecwall

Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @ericharmeling, @knz, @msbutler, and @yuzefovich)


pkg/sql/crdb_internal.go line 8609 at r8 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

I added a skip for local-mixed-22_2-23_1 in two places. This means that SHOW SYSTEM GRANTS might not work in mixed version 22.2-23.2 cluster when issued against the older node (apparently, we'll have experimental support for this). The alternative is to change the schema to remove user_id column, but this would make the virtual table schema differ from the system table, so I chose the skip option.

we no longer need to support mixed 22.2/23.2 clusters, so i guess that's why we didn't end up needing a change here?


pkg/sql/privilege/kind.go line 30 at r10 (raw file):

	CREATE Kind = 2
	DROP   Kind = 3
	// This is a placeholder to make sure that 4 is not reused.

actually, i think it might be safe to reuse 4 at this point. i guess no need to try that unless there's a need.


pkg/sql/privilege/privilege.go line 136 at r4 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

Want to bring attention to this change - previously we had privilege.List.String() which joined strings with ", " separator, we removed that implementation, so in some logic tests we started seeing a different error message where multiple privileges were joined just by a comma. Adding a space here should fix those tests, but it also seems like a nice change on its own. Thoughts?

i agree, thanks


pkg/sql/logictest/testdata/logic_test/grant_database line 12 at r7 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

Here is another minor difference - we now use privilege.List.SortedDisplayNames in a few places which returns []string, and when that gets stringified, we see square brackets around the names. Seems reasonable.

+1

@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch from b3b2c63 to 1c6839a Compare October 20, 2023 18:14
In a way akin to 56e07c1, this change
introduces a distinction between the *name* of a system privilege,
which is the one facing end-users; and the *internal key* of the
privilege, which is stored in `system.privileges`.

This distinction exists to allow us to rename the user-facing
privileges towards better UX and clearer docs without having to
rewrite the contents of `system.privileges`.

The internal key is meant to remain immutable after a privilege is
added in the source code. For privileges that will be added in the
future, we can now even envision storing numbers or UUIDs instead in
`system.privileges`.

For backward-compatibility, the internal key for a system privilege is
also recognized as input in SQL syntax.

(An argument could be made that we would be better served by
`system.privilege` containing an array of integers with the internal
IDs, but this change aims to avoid _any_ migration whatsoever, and
currently the system table already contains strings.)

Release note: None
@yuzefovich yuzefovich force-pushed the 20231003-priv-ids-vs-names branch from 1c6839a to ce64d8b Compare October 20, 2023 20:08
Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

bors r+

Reviewed 11 of 11 files at r11, 3 of 3 files at r12, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall, @ericharmeling, @msbutler, and @rafiss)


pkg/sql/crdb_internal.go line 8609 at r8 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

we no longer need to support mixed 22.2/23.2 clusters, so i guess that's why we didn't end up needing a change here?

Right, we no longer need the skip, so I removed a couple from the logic tests.


pkg/sql/privilege/kind.go line 30 at r10 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

actually, i think it might be safe to reuse 4 at this point. i guess no need to try that unless there's a need.

Ack.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Oct 21, 2023

Build succeeded:

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Oct 21, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error setting reviewers, but backport branch blathers/backport-release-23.2-111653 is ready: POST https://api.github.com/repos/cockroachdb/cockroach/pulls/112810/requested_reviewers: 422 Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the cockroachdb/cockroach repository. []

Backport to branch 23.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-23.2.x PAST MAINTENANCE SUPPORT: 23.2 patch releases via ER request only

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants