spanconfig: teach the KVAccessor about system span configurations#76414
spanconfig: teach the KVAccessor about system span configurations#76414craig[bot] merged 2 commits intocockroachdb:masterfrom
Conversation
da946c9 to
d8d71c1
Compare
d8d71c1 to
44c2d1f
Compare
irfansharif
left a comment
There was a problem hiding this comment.
LGTM (looked at only the last commit here).
| var spanRe = regexp.MustCompile(`^\[(\w+),\s??(\w+)\)$`) | ||
|
|
||
| // systemTargetRe matches strings of the form | ||
| // "{source=(<id>|system),target=(<id>|system)}". |
There was a problem hiding this comment.
Simplify the grammar to drop "system". There are already too many "system" in our vocabulary and here used for two different meanings ("system tenant" and "system target"); there's no difference between tenant-id==1 and system, and the test cases render them differently. It would help simplify your parser below. I'd be ok exporting it now too (ParseSystemTarget) like we do for everything else -- we are going to use something like it in spanconfig.store tests shortly anyway.
| default: | ||
| t.Fatalf("expected %s to match span or system target regex", target) | ||
| } | ||
| return spanconfig.Target{} |
There was a problem hiding this comment.
Could panic instead of this confusing return.
| case target.IsSystemTarget(): | ||
| return printSystemTarget(target.GetSystemTarget()) | ||
| default: | ||
| t.Fatalf("unknown traget type") |
There was a problem hiding this comment.
Typo: traget. Could panic below instead of the empty string return.
| // Duplicate in toDelete with some span targets. | ||
| toDelete: []spanconfig.Target{ | ||
| spanconfig.MakeTargetFromSpan(roachpb.Span{Key: roachpb.Key("a"), EndKey: roachpb.Key("c")}), | ||
| makeTenantClusterTarget(roachpb.SystemTenantID.InternalValue), |
There was a problem hiding this comment.
(Applies to earlier commits as well I think) Should we be using InternalValue? The name and surrounding comment suggests we shouldn't:
cockroach/pkg/roachpb/data.proto
Lines 752 to 753 in 5a18c3a
There was a problem hiding this comment.
Could also make our helper take the TenantID type instead. If we got rid of type casting errors like suggested in the other review, some of this code could be simplified/inlined given there'd be no error handling needed.
| ok | ||
|
|
||
| kvaccessor-get | ||
| system-target {source=system,target=10} |
There was a problem hiding this comment.
How exactly are we expecting the reconciler to get all system targets down the line? The API here suggests that each key is to be retrieved by naming it one-by-one. Will that continue to be the case?
44c2d1f to
66916bf
Compare
arulajmani
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @adityamaru, @ajwerner, @irfansharif, and @nvanbenschoten)
pkg/spanconfig/spanconfigtestutils/utils.go, line 36 at r4 (raw file):
Previously, irfansharif (irfan sharif) wrote…
Simplify the grammar to drop "system". There are already too many "system" in our vocabulary and here used for two different meanings ("system tenant" and "system target"); there's no difference between tenant-id==1 and system, and the test cases render them differently. It would help simplify your parser below. I'd be ok exporting it now too (ParseSystemTarget) like we do for everything else -- we are going to use something like it in spanconfig.store tests shortly anyway.
I agree, done. I only added it because that's how we stringify tenant IDs, but I instead ended up changing the String method on system targets instead. As for making it public, I agree we'll need it very soon, but I'll just do it then.
pkg/spanconfig/spanconfigtestutils/utils.go, line 98 at r4 (raw file):
Previously, irfansharif (irfan sharif) wrote…
Could panic instead of this confusing return.
Done.
pkg/spanconfig/spanconfigtestutils/utils.go, line 263 at r4 (raw file):
Previously, irfansharif (irfan sharif) wrote…
Typo: traget. Could panic below instead of the empty string return.
Done.
pkg/spanconfig/spanconfigkvaccessor/testdata/system_span_configs, line 72 at r4 (raw file):
Previously, irfansharif (irfan sharif) wrote…
How exactly are we expecting the reconciler to get all system targets down the line? The API here suggests that each key is to be retrieved by naming it one-by-one. Will that continue to be the case?
I was going to follow this PR with one that adds another method to the KVAccessor, something of the form GetAllHostTenantInstalledSystemSpanConfigs which wouldn't be accessible on the connector like we talked offline.
pkg/spanconfig/spanconfigkvaccessor/validation_test.go, line 155 at r4 (raw file):
Previously, irfansharif (irfan sharif) wrote…
Could also make our helper take the TenantID type instead. If we got rid of type casting errors like suggested in the other review, some of this code could be simplified/inlined given there'd be no error handling needed.
This just goes away once I rebased from the last PR. Good call on the internal value stuff -- tacked on a commit to fix that.
66916bf to
51246a3
Compare
|
|
||
| # Test with an empty slate. | ||
| kvaccessor-get | ||
| system-target {cluster} |
There was a problem hiding this comment.
I'd recommend getting rid of this part of the grammar too. It's not adding anything that isn't represented by {source=1}, something our grammar can already represent. I also have misgivings about spanconfig.MakeClusterTarget. For one, despite the same name, it's wholly different conceptually to ptpb.MakeClusterTarget. We chatted a bit offline about this with @adityamaru, regardless of where we land, simplifying the testdata grammar itself is a good thing IMO.
| line = strings.TrimPrefix(line, systemTargetPrefix) | ||
| default: | ||
| t.Fatalf( | ||
| "malformed line %q, expected to find spanPrefix %q or system target prefix %q", |
There was a problem hiding this comment.
[nit] "expected to find %q or %q prefix" would avoid stuttering in your error message.
| // MakeSystemTarget constructs, validates, and returns a new SystemTarget. | ||
| func MakeSystemTarget( | ||
| sourceTenantID roachpb.TenantID, targetTenantID *roachpb.TenantID, | ||
| sourceTenantID roachpb.TenantID, targetTenantID roachpb.TenantID, |
There was a problem hiding this comment.
For future PRs: worth reconsidering IMO these constructors, in addition to MakeClusterTarget. My preference:
MakeClusterTarget(roachpb.TenantID) SystemTarget // takes in what you call today "source"
MakeTenantsTarget([]roachpb.TenantID) []SystemTarget // doesn't take in the source, has to be system tenant, but takes in dests
Pick whatever names you want for these.
There was a problem hiding this comment.
What we ended up with for the time being is:
MakeClusterTarget() SystemTarget // returns a system target with only the sourceTarget set to the system tenant ID
MakeTenantTarget(source, target roachpb.TenantID)
Then the SQLTranslator will:
// Aggregate cluster target protections for the tenant.
clusterProtections := ptsStateReader.getProtectionPoliciesForCluster()
if len(clusterProtections) != 0 {
var systemTarget spanconfig.SystemTarget
var err error
if sourceTenantID == roachpb.SystemTenantID {
systemTarget = spanconfig.MakeClusterTarget()
} else {
systemTarget, err = spanconfig.MakeTargetTenant(sourceTenantID, sourceTenantID)
if err != nil {
return nil, err
}
}
clusterSystemRecord := spanconfig.Record{
Target: spanconfig.MakeTargetFromSystemTarget(systemTarget),
Config: roachpb.SpanConfig{GCPolicy: roachpb.GCPolicy{ProtectionPolicies: clusterProtections}}}
records = append(records, clusterSystemRecord)
}
// Aggregate tenant target protections.
tenantProtections := ptsStateReader.getProtectionPoliciesForTenants()
for _, protection := range tenantProtections {
tenantProtection := protection
tenantSystemRecord := spanconfig.Record{
Target: spanconfig.MakeTargetFromSystemTarget(spanconfig.MakeTenantTarget(
sourceTenantID, tenantProtection.tenantID)),
Config: roachpb.SpanConfig{GCPolicy: roachpb.GCPolicy{
ProtectionPolicies: tenantProtection.protections}},
}
records = append(records, tenantSystemRecord)
}
There was a problem hiding this comment.
But we could make it simpler without complicating the client side:
// Aggregate cluster target protections for the tenant.
clusterProtections := ptsStateReader.getProtectionPoliciesForCluster()
if len(clusterProtections) != 0 {
clusterSystemRecord := spanconfig.Record{
Target: spanconfig.MakeTargetFromSystemTarget(spanconfig.MakeClusterTarget(sourceTenantID)),
Config: roachpb.SpanConfig{GCPolicy: roachpb.GCPolicy{ProtectionPolicies: clusterProtections}}}
records = append(records, clusterSystemRecord)
}
// Aggregate tenant target protections.
tenantProtections := ptsStateReader.getProtectionPoliciesForTenants()
for _, protection := range tenantProtections {
tenantProtection := protection
tenantSystemRecord := spanconfig.Record{
Target: spanconfig.MakeTargetFromSystemTarget(spanconfig.MakeTenantTarget(tenantProtection.tenantID)),
Config: roachpb.SpanConfig{GCPolicy: roachpb.GCPolicy{
ProtectionPolicies: tenantProtection.protections}},
}
records = append(records, tenantSystemRecord)
}There was a problem hiding this comment.
If we want to simplify the client side logic, it might be worth considering changing the ptsStateReader to return ptpb.Targets instead of protection policies like it does today. Then, we could write conversions to/from ptpb.Targets to spanconfig.Targets.
There was a problem hiding this comment.
It's not about simplifying client code, it's about concepts. We have a cluster concept in backup/pts, and we have another cluster concept here that means a wholly different thing. They both have the same MakeClusterTarget signature too. It's a bit difficult to see why.
51246a3 to
fd5aa86
Compare
This patch teaches the KVAccessor to update and get system span configurations. Release note: None
Release note: None
fd5aa86 to
f78be51
Compare
arulajmani
left a comment
There was a problem hiding this comment.
TFTR!
bors r+
Dismissed @irfansharif from a discussion.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @adityamaru, @ajwerner, @irfansharif, and @nvanbenschoten)
pkg/spanconfig/spanconfigtestutils/utils.go, line 157 at r8 (raw file):
Previously, irfansharif (irfan sharif) wrote…
[nit] "expected to find %q or %q prefix" would avoid stuttering in your error message.
Done.
pkg/spanconfig/spanconfigkvaccessor/testdata/system_span_configs, line 5 at r8 (raw file):
Previously, irfansharif (irfan sharif) wrote…
I'd recommend getting rid of this part of the grammar too. It's not adding anything that isn't represented by {source=1}, something our grammar can already represent. I also have misgivings about spanconfig.MakeClusterTarget. For one, despite the same name, it's wholly different conceptually to ptpb.MakeClusterTarget. We chatted a bit offline about this with @adityamaru, regardless of where we land, simplifying the testdata grammar itself is a good thing IMO.
I agree that the cluster target is just {source=1}, but I think having this {cluster} here does a bit for readability.
@adityamaru captured where the discussion around MakeClusterTarget went above.
|
Build succeeded: |
First 3 commits are from #76219, this one's quite small -- mostly just tests.
This patch teaches the KVAccessor to update and get system span
configurations.
Release note: None