config: introduce pseudo "tenants" zone#49784
Conversation
00de2c4 to
29a399d
Compare
asubiotto
left a comment
There was a problem hiding this comment.
Glad to see this! Do we have an issue to track post phase 2 work for tenant zone configs once #49445 is closed? We should create one if not.
I'll rerun the logic tests to see which blacklists can be removed.
Reviewed 1 of 1 files at r1, 4 of 4 files at r2, 5 of 5 files at r3, 44 of 44 files at r4.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @dt, @nvanbenschoten, and @tbg)
pkg/sql/drop_table.go, line 354 at r4 (raw file):
// this faster mechanism. if tableDesc.IsTable() && !tableDesc.IsInterleaved() { // TODO BEFORE MERGING: confirm that we can delete this.
I think so. Based on 7abac2f, this code was added as a sanity check. OOC, how would this inability to get a zone config for a dropped tenant table have manifested itself otherwise? Would the default TTL be applied?
pkg/sql/logictest/testdata/logic_test/tenant_unsupported, line 37 at r4 (raw file):
# https://github.com/cockroachdb/cockroach/issues/49318 is fixed. # statement error operation is unsupported # ALTER TABLE kv CONFIGURE ZONE USING num_replicas = 123
You can probably delete this.
tbg
left a comment
There was a problem hiding this comment.
Now that secondary tenant ranges have a zone config to call their own, we can make sense of calls from SQL into the zone configuration infrastructure. We gate off calls that don't make sense for secondary tenants and clean up hooks in SQL that handle zone config manipulation.
This left me slightly confused. Secondary tenants don't really get to do anything with the zone configs, or am I missing something? All they will see is the default zone config (configured at startup). Your description made it sound as if tenants can do... something with the zone config, but I don't think you even let them list the "background" config that applies to them.
Reviewed 1 of 1 files at r1, 4 of 4 files at r2, 5 of 5 files at r3, 44 of 44 files at r4.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @dt and @nvanbenschoten)
pkg/config/system.go, line 207 at r3 (raw file):
// No maximum specified; maximum ID is the last entry in the descriptor // table.
or the largest pseudo ID, whichever is larger.
pkg/config/system.go, line 221 at r3 (raw file):
// Maximum specified: need to search the descriptor table. Binary search // through all descriptor table values to find the first descriptor with ID // >= maxID.
and pick either it or maxPseudoID.
pkg/config/system.go, line 229 at r3 (raw file):
} var id uint32 id, err = keys.TODOSQLCodec.DecodeDescMetadataID(searchSlice[i].Key)
This can't be intentional, at least not without a comment.
pkg/keys/constants.go, line 399 at r4 (raw file):
// PseudoTableIDs is the list of ids from above that are not real tables (i.e. // there's no table descriptor). They're grouped here because the cluster // bootstrap process needs to create splits for them; splits for the tables
Can you revisit the "because the bootstrap process needs to create splits for them"? I think I remember that it does create splits for the tables with these IDs (which don't exist). I think something is fishy here, it's not like this package has any notion of where the split points that correspond to these pseudoIDs are. Probably the comment makes no sense.
pkg/sql/zone_config.go, line 159 at r2 (raw file):
func ZoneConfigHook( cfg *config.SystemConfig, id uint32, ) (*zonepb.ZoneConfig, *zonepb.ZoneConfig, bool, error) {
While you're here, explain the retvals?
29a399d to
ea2c18b
Compare
nvb
left a comment
There was a problem hiding this comment.
TFTRs!
Do we have an issue to track post phase 2 work for tenant zone configs once #49445 is closed?
I opened #49854 to track this.
This left me slightly confused.
Reworded.
Reviewable status:
complete! 1 of 0 LGTMs obtained (waiting on @asubiotto, @dt, and @tbg)
pkg/config/system.go, line 207 at r3 (raw file):
Previously, tbg (Tobias Grieger) wrote…
or the largest pseudo ID, whichever is larger.
Done.
pkg/config/system.go, line 221 at r3 (raw file):
Previously, tbg (Tobias Grieger) wrote…
and pick either it or maxPseudoID.
Done.
pkg/config/system.go, line 229 at r3 (raw file):
Previously, tbg (Tobias Grieger) wrote…
This can't be intentional, at least not without a comment.
This was a drive-by fix. The code was subtly broken before. We could hit an error while decoding and then overwrite it on the next iteration. I added a test that tests a case that would have previously been handled incorrectly.
pkg/keys/constants.go, line 399 at r4 (raw file):
Previously, tbg (Tobias Grieger) wrote…
Can you revisit the "because the bootstrap process needs to create splits for them"? I think I remember that it does create splits for the tables with these IDs (which don't exist). I think something is fishy here, it's not like this package has any notion of where the split points that correspond to these pseudoIDs are. Probably the comment makes no sense.
I'm not sure I understand the ask. We do create splits for these tables, even though they don't really exist. Do you take issue with the keys package mentioning split points?
pkg/sql/drop_table.go, line 354 at r4 (raw file):
Previously, asubiotto (Alfonso Subiotto Marqués) wrote…
I think so. Based on 7abac2f, this code was added as a sanity check. OOC, how would this inability to get a zone config for a dropped tenant table have manifested itself otherwise? Would the default TTL be applied?
Thanks for confirming. It looks like an inability to get a zone config for a dropped tenant table will now manifest as an error being thrown by updateStatusForGCElements from the call to GetZoneConfigForObject. I don't think there's ever a case where a table wouldn't have an associated zone config though, and David seemed to think this was pointless, so I'm going to remove it.
pkg/sql/zone_config.go, line 159 at r2 (raw file):
Previously, tbg (Tobias Grieger) wrote…
While you're here, explain the retvals?
Done.
pkg/sql/logictest/testdata/logic_test/tenant_unsupported, line 37 at r4 (raw file):
Previously, asubiotto (Alfonso Subiotto Marqués) wrote…
You can probably delete this.
Done.
asubiotto
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 2 of 0 LGTMs obtained (waiting on @asubiotto, @dt, and @tbg)
tbg
left a comment
There was a problem hiding this comment.
Reviewed 47 of 47 files at r5, 4 of 4 files at r6, 5 of 5 files at r7, 3 of 3 files at r8, 44 of 44 files at r9.
Reviewable status:complete! 2 of 0 LGTMs obtained (waiting on @asubiotto, @dt, and @tbg)
18df388 to
56f002c
Compare
56f002c to
f59c561
Compare
We don't return nil values from MVCCScan. Also, a bit of harmless cleanup.
Mostly cleaning up comments. Noticed while re-familiarizing myself with this package.
This was not behaving correctly, and was breaking things when a pseudo table had the largest ID below MaxReservedDescID.
This was wrong before. Adds testing.
f59c561 to
85e4ac4
Compare
Fixes cockroachdb#49318. Fixes cockroachdb#49445. Progress towards cockroachdb#48123. Informs cockroachdb#48774. This commit introduces a new pseudo object ID in the system tenant's namespace called "tenants". Like "liveness" and "timeseries" before it, the pseudo object allows zone configurations to be applied to pseudo-objects that do not live directly in the system tenant's SQL keyspace. In this case, the "tenants" zone allows zone configurations to be set by the system tenant and applied to all other tenants in the system. There may come a time when we want secondary tenants to have more control over their zone configurations, but that will require a much larger change to the zone config structure and UX as a whole. While making this change, we rationalize the rest of zone configuration handling and how it relates to multi-tenancy. Now that secondary tenant ranges have a zone config to call their own, we can make sense of calls from KV into the zone configuration infrastructure. We gate off calls that don't make sense for secondary tenants and clean up hooks in SQL that handle zone config manipulation. All of this works towards a good cause - we eliminate the remaining uses of `keys.TODOSQLCodec` from `pkg/sql/...` and `pkg/config/...`, bringing us a big step closer towards being able to remove the placeholder and close cockroachdb#48123. This work also reveals that in order to address cockroachdb#48774, we need to be able to determine split points from the SystemConfig. This makes it very difficult to split on secondary tenant object (e.g. table) boundaries. However, it makes it straightforward to split on secondary tenant keysapce boundaries. This is already what we were thinking (see cockroachdb#47907), so out both convenience and desire, I expect that we'll follow this up with a PR that splits Ranges only at secondary tenant boundaries - placing the overhead of an otherwise empty tenant at only a single Range and a few KBs of data.
85e4ac4 to
6953182
Compare
|
bors r+ |
Build succeeded |
Fixes #49318.
Fixes #49445.
Progress towards #48123.
Informs #48774.
This commit introduces a new pseudo object ID in the system tenant's namespace called "tenants". Like "liveness" and "timeseries" before it, the pseudo object allows zone configurations to be applied to pseudo-objects that do not live directly in the system tenant's SQL keyspace. In this case, the "tenants" zone allows zone configurations to be set by the system tenant and applied to all other tenants in the system. There may come a time when we want secondary tenants to have more control over their zone configurations, but that will require a much larger change to the zone config structure and UX as a whole.
While making this change, we rationalize the rest of zone configuration handling and how it relates to multi-tenancy. Now that secondary tenant ranges have a zone config to call their own, we can make sense of calls from SQL into the zone configuration infrastructure. We gate off calls that don't make sense for secondary tenants and clean up hooks in SQL that handle zone config manipulation.
All of this works towards a good cause - we eliminate the remaining uses of
keys.TODOSQLCodecfrompkg/sql/...andpkg/config/..., bringing us a big step closer towards being able to remove the placeholder and close #48123.This work also reveals that in order to address #48774, we need to be able to determine split points from the SystemConfig. This makes it very difficult to split on secondary tenant object (e.g. table) boundaries. However, it makes it straightforward to split on secondary tenant keysapce boundaries. This is already what we were thinking (see #47907), so out both convenience and desire, I expect that we'll follow this up with a PR that splits Ranges only at secondary tenant boundaries - placing the overhead of an otherwise empty tenant at only a single Range and a few KBs of data.