descs,catkv: rewrite Collection backend#87067
Conversation
8dd5006 to
eebff4d
Compare
eebff4d to
baefec5
Compare
ajwerner
left a comment
There was a problem hiding this comment.
This is nice. The only major hesitation I have is around the SystemDatabaseCache. The rest of my comments are minor.
Reviewed 1 of 1 files at r1, 34 of 34 files at r2, all commit messages.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @postamar and @rhu713)
pkg/sql/catalog/descs/descriptor.go line 225 at r2 (raw file):
// // TODO(ajwerner): More generally leverage this set of kv descriptors on // the resolution path.
I think I'd say Jason achieved this TODO.
Code quote:
// TODO(ajwerner): More generally leverage this set of kv descriptors on
// the resolution path.pkg/sql/catalog/descs/descriptor.go line 338 at r2 (raw file):
// and (2) if mutable descriptors are requested, these are present in the // uncommitted descriptors layer. // Known validation levels can be provided via validationLevels, otherwise when
nit: my brain had a hard time parsing otherwise when nil safe defaults are used instead. Can you sprinkle in a comma or two?
Code quote:
// Known validation levels can be provided via validationLevels, otherwise when
// nil safe defaults are used instead. In any case, after validation ispkg/sql/catalog/descs/descriptor.go line 348 at r2 (raw file):
validationLevels []catalog.ValidationLevel, ) error { if validationLevels == nil {
add a defensive check that if validationLevels is not nil that it has the same length as descs?
pkg/sql/catalog/descs/uncommitted_descriptors.go line 181 at r2 (raw file):
return errors.Wrap(err, "Memory usage exceeds limit for uncommittedDescriptors") } ud.mutable.Upsert(mut, true /* skipNameMap */)
why are we unconditionally skipping the name map? It feels like if we're going to always skip the nameMap we should use a data structure that does not have one.
pkg/sql/catalog/internal/catkv/stored_catalog.go line 38 at r2 (raw file):
// cache mirrors the descriptors in storage. // This map does not store descriptors by name.
I said it elsewhere: let's export an object from nstree which wraps the existing byIDMap and avoid the confusion this thing is causing
pkg/sql/catalog/internal/catkv/system_database_cache.go line 34 at r2 (raw file):
// the lease manager to cache all system table IDs. type SystemDatabaseCache struct { syncutil.RWMutex
I don't think it's right to export the mutex methods. The style here is generally to embed the mutex inside of an anonymous, embedded struct. I think that would work nicely here.
type SystemDatabaseCache struct {
mu struct {
syncutil.RWMutex
m map[roachpb.Version]*nstree.MutableCatalog
}
}
pkg/sql/catalog/internal/catkv/system_database_cache.go line 35 at r2 (raw file):
type SystemDatabaseCache struct { syncutil.RWMutex m map[roachpb.Version]*nstree.MutableCatalog
The semantics of the versions in use here are interesting. I think that right now, it doesn't really matter because we never rename something in the system database and we only use this for caching names, right?
pkg/sql/catalog/internal/catkv/system_database_cache.go line 107 at r2 (raw file):
// Anything not pertaining to the system database is ignored. // Presently, system descriptors are all ignored and only namespace entries are // used.
what's your vision here? My plan for 23.1 is to start treating system descriptors, to the extent possible, like regular descriptors and not to make them more special. We'll need this to allow for deeper configuration of multi-region primitives
Code quote:
// Presently, system descriptors are all ignored and only namespace entries are
// used.Recent changes to the descriptors collection have uncovered some brittleness in the logic for writing namespace entries for dropped descriptors when creating backups. This commit makes it: - write deletion entries for descriptors other than tables, preventing possible catalog corruption. - throw a better error message when attempting to write two identical keys consecutively. Release justification: low-risk change Release note: None
postamar
left a comment
There was a problem hiding this comment.
Thanks for the review.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @postamar, and @rhu713)
pkg/sql/catalog/descs/descriptor.go line 225 at r2 (raw file):
Previously, ajwerner wrote…
I think I'd say Jason achieved this TODO.
Removing.
pkg/sql/catalog/descs/descriptor.go line 338 at r2 (raw file):
Previously, ajwerner wrote…
nit: my brain had a hard time parsing
otherwise when nil safe defaults are used instead. Can you sprinkle in a comma or two?
Rewriting.
pkg/sql/catalog/descs/descriptor.go line 348 at r2 (raw file):
Previously, ajwerner wrote…
add a defensive check that if
validationLevelsis notnilthat it has the same length asdescs?
Will do.
pkg/sql/catalog/descs/uncommitted_descriptors.go line 181 at r2 (raw file):
Previously, ajwerner wrote…
why are we unconditionally skipping the name map? It feels like if we're going to always skip the nameMap we should use a data structure that does not have one.
Will do.
pkg/sql/catalog/internal/catkv/stored_catalog.go line 38 at r2 (raw file):
Previously, ajwerner wrote…
I said it elsewhere: let's export an object from
nstreewhich wraps the existingbyIDMapand avoid the confusion this thing is causing
Will do. Agreed.
pkg/sql/catalog/internal/catkv/system_database_cache.go line 34 at r2 (raw file):
Previously, ajwerner wrote…
I don't think it's right to export the mutex methods. The style here is generally to embed the mutex inside of an anonymous, embedded struct. I think that would work nicely here.
type SystemDatabaseCache struct { mu struct { syncutil.RWMutex m map[roachpb.Version]*nstree.MutableCatalog } }
Whoops, right. Will do.
pkg/sql/catalog/internal/catkv/system_database_cache.go line 35 at r2 (raw file):
Previously, ajwerner wrote…
The semantics of the versions in use here are interesting. I think that right now, it doesn't really matter because we never rename something in the system database and we only use this for caching names, right?
Indeed it's just me being defensive. Adding commentary.
pkg/sql/catalog/internal/catkv/system_database_cache.go line 107 at r2 (raw file):
Previously, ajwerner wrote…
what's your vision here? My plan for 23.1 is to start treating system descriptors, to the extent possible, like regular descriptors and not to make them more special. We'll need this to allow for deeper configuration of multi-region primitives
My vision is the same as yours. Adding commentary to explain why we don't actually cache system descriptors other than the system DB.
3119e59 to
803ab34
Compare
This rewrite was motivated by the upcoming need to validate descriptors up to different levels depending on why they were being read. Doing so requires tracking the validation level in a more granular fashion within the Collection's caches. Trying to make this work within the existing scheme where the uncommitted and stored descriptors are all in the same layer turned out to be painful and hard to reason about. This commit does several things: 1. The uncommitted descriptors are spun off into their own distinct layer again. 2. The stored descriptors now mirror what's in storage and so that whole layer could effectively be moved to catkv. 3. Execution flow through the layers in the Collection is now straightforwardly implemented and easy to reason about. Validation and hydration now take place in unique, well-defined steps at the very end of a lookup, be it by name or by ID. 3. Instances of this new stored descriptors layer are now also used by the lease manager, the Direct() interface provided by the Collection, etc. 4. As a result the logic in catkv could be significantly cleaned up. In particular, lookups for descriptor validation are significantly less convoluted now. 5. The SystemNamespaceCache object was also moved to catkv, renamed to SystemDatabaseCache and now caches based on the cluster version. Despite significant rewriting of the inner workings of the descriptors collection, this commit should not functionally change anything. Informs cockroachdb#85263. Release justification: low-risk, high-benefit change. Release note: None
This commit adds a new nstree.Map-like data structure without any by-name operations. Release justification: low-risk change which does not change functionality Release note: None
803ab34 to
c74da8a
Compare
ajwerner
left a comment
There was a problem hiding this comment.
my comments are nits, but in general, I'd prefer we not export methods unless we need them to conform to some interface explicitly. I spent a little while looking around for such an interface.
The internal executor change I sort of get, but I don't fully get. I feel like we can tighten up the code and commentary around that change.
Reviewed 3 of 34 files at r3, 19 of 34 files at r4, 8 of 17 files at r5, 1 of 2 files at r6, 9 of 11 files at r7, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @ajwerner, @postamar, and @rhu713)
which it never has when using the planner's state.
Is that true? Consider:
Line 2641 in c6f2617
Line 2436 in c6f2617
Line 298 in 17cfa7c
Code quote:
prevent that by attaching the logic to reset the synthetic descriptors to
whether the internal executor has attached synthetic descriptors, which
it never has when using the planner's state.
-- commits line 5 at r7:
I anticipate this all worked because the backups don't store dropped descriptors. Does this commit fix any known bug? Should it be in this PR or should it be pulled out into a separate one such that it might be tested?
Code quote:
Recent changes to the descriptors collection have uncovered some
brittleness in the logic for writing namespace entries for dropped
descriptors when creating backups.
pkg/sql/internal.go line 337 at r6 (raw file):
ex.extraTxnState.descCollection = ie.extraTxnState.descCollection ex.extraTxnState.fromOuterTxn = true ex.extraTxnState.shouldResetSyntheticDescriptors = len(ie.syntheticDescriptors) > 0
can you add more commentary here?
pkg/sql/internal.go line 334 at r7 (raw file):
// and job collections from the caller. postSetupFn := func(ex *connExecutor) { if ie.extraTxnState != nil && ie.extraTxnState.descCollection != nil {
what's the case where extraTxnState is not nil but descCollection is? Also, while you're here, if you have to do another revision, rename to descsCollection?
pkg/sql/catalog/descs/synthetic_descriptors.go line 23 at r7 (raw file):
} func (sd *syntheticDescriptors) Add(desc catalog.Descriptor) {
why are all these methods now exported?
pkg/sql/catalog/descs/uncommitted_descriptors.go line 63 at r7 (raw file):
// Reset zeroes the object for re-use in a new transaction. func (ud *uncommittedDescriptors) Reset(ctx context.Context) {
Why are all of the methods on this unexported struct exported? It doesn't seem like it's embedded anywhere
0686a75 to
c3af318
Compare
…otrs The pre-existing logic would reset the synthetic descriptors whenever anything was run on behalf of the internal planner. That's not what we want. We prevent that by attaching the logic to reset the synthetic descriptors to whether the internal executor has attached synthetic descriptors, which it never has when using the declarative schema changer. Release justification: part of bug fix Release note: None
This allows synthetic descriptors to show up in virtual tables. Release justification: part of bug fix. Release note: None
This commit doesn't otherwise change anything. Release justification: very low-risk change Release note: None
c3af318 to
286d828
Compare
|
Thanks for the review! I unexported the methods and other such things. bors r+ |
|
Build succeeded: |
86420: catalog: spin off and adopt back-reference validation level r=postamar a=postamar Previously, forward- and backward-reference-checking was done in one same validation phase, identified by the catalog.ValidationLevelCrossReferences constant and a similarly named method on catalog.Descriptor. This validation phase was exercised both when reading and when writing descriptors. The problem with this was that a corrupt back-reference would make a table unusable until descriptor surgery was performed. While this might be warranted in the context of schema changes, that's not the case when it comes to queries, which don't really care about these back-references. For this reason, we want to bypass back-reference validation when reading descriptors for the purpose of serving queries. These reads are characterized by the flag AvoidLeased being unset. To do this, this commit splits the cross-reference validation level into two. The back-reference level now exists to check the integrity of back-references directly, as well as to check that forward references also have matching back-references in the referenced descriptors. Fixes #85263. Release justification: low-risk, high benefit change Release note: None ---- This PR is based on #87067 which is the product of what made sense for me to take action on right now regarding #86420 (comment) . Co-authored-by: Marius Posta <marius@cockroachlabs.com>
Recent work on the descs.Collection (PR cockroachdb#87067) introduced a regression in which it would return bad lease.IDVersion versions for the uncommitted descriptors. Fixes cockroachdb#87672. Release justification: important bug fix Release note: None
Recent work on the descs.Collection (PR cockroachdb#87067) introduced a regression in which it would return bad lease.IDVersion versions for the uncommitted descriptors. In addition to fixing this, this commit corrects the lifecycle of the `original` descriptors in the uncommitted layer, which now remain unaffected by changes in the storage layer. This commit tightens the constraints on what can be passed to AddUncommittedDescriptor. Fixes cockroachdb#87672. Release justification: important bug fix Release note: None
87525: sql: fix recording db names with hyphen for idx usage stats r=xinhaoz a=xinhaoz Fixes #85577 This commit fixes a bug where the query constructed during index usage stats recording failed for db names containing a hyphen. The db name placeholder is now converted to string from the `tree.Name`, which should properly escape hyphen characters for the query. Release justification: bug fix Release note (bug fix): index usage stats are properly captured for database names with hyphens 87706: descs: fix txn commit waiting on wrong lease version r=postamar a=postamar Recent work on the descs.Collection (PR #87067) introduced a regression in which it would return bad lease.IDVersion versions for the uncommitted descriptors. Fixes #87672. Release justification: important bug fix Release note: None Co-authored-by: Xin Hao Zhang <xzhang@cockroachlabs.com> Co-authored-by: Marius Posta <marius@cockroachlabs.com>
Recent work on the descs.Collection (PR #87067) introduced a regression in which it would return bad lease.IDVersion versions for the uncommitted descriptors. In addition to fixing this, this commit corrects the lifecycle of the `original` descriptors in the uncommitted layer, which now remain unaffected by changes in the storage layer. This commit tightens the constraints on what can be passed to AddUncommittedDescriptor. Fixes #87672. Release justification: important bug fix Release note: None
This rewrite was motivated by the upcoming need to validate descriptors
up to different levels depending on why they were being read. Doing so
requires tracking the validation level in a more granular fashion within
the Collection's caches. Trying to make this work within the existing
scheme where the uncommitted and stored descriptors are all in the same
layer turned out to be painful and hard to reason about.
This commit does several things:
again.
layer could effectively be moved to catkv.
straightforwardly implemented and easy to reason about. Validation and
hydration now take place in unique, well-defined steps at the very end
of a lookup, be it by name or by ID.
the lease manager, the Direct() interface provided by the Collection,
etc.
In particular, lookups for descriptor validation are significantly
less convoluted now.
supports caching descriptors, and so was renamed to
SystemCatalogCache.
Despite significant rewriting of the inner workings of the descriptors
collection, this commit should not functionally change anything.
Informs #85263.
Release justification: low-risk, high-benefit change.
Release note: None