sql: use declarative schemachange for add column sequence exprs#81782
sql: use declarative schemachange for add column sequence exprs#81782craig[bot] merged 1 commit intocockroachdb:masterfrom
Conversation
5e63e21 to
a0836b4
Compare
Xiang-Gu
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @fqazi and @Xiang-Gu)
pkg/sql/schemachanger/scexec/exec_backfill.go line 256 at r1 (raw file):
}); err != nil { // We ran into an uncategorized schema change error. deps.Telemetry().IncrementSchemaChangeErrorType("uncategorized")
so all we need to do is add a Telemetry interface for declarative schema changer, and invoke it during backfill if the backfill fails? I saw only the "uncategorized" error. Are there any other types of errors?
ajwerner
left a comment
There was a problem hiding this comment.
There's something funny going on in the restore case
Reviewed 6 of 7 files at r1, all commit messages.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @fqazi, and @Xiang-Gu)
-- commits line 17 at r1:
detritus
0493186 to
f703132
Compare
f703132 to
01c2118
Compare
fqazi
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @dt, and @Xiang-Gu)
Previously, ajwerner wrote…
detritus
Done.
pkg/sql/schemachanger/scexec/exec_backfill.go line 256 at r1 (raw file):
Previously, Xiang-Gu (Xiang Gu) wrote…
so all we need to do is add a Telemetry interface for declarative schema changer, and invoke it during backfill if the backfill fails? I saw only the "uncategorized" error. Are there any other types of errors?
The only other type we have is for constraint validation failures. These can be extended later on
7e8fda1 to
f9bd08e
Compare
ajwerner
left a comment
There was a problem hiding this comment.
This is very close, nice
Reviewed 2 of 9 files at r3.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @dt, @fqazi, and @Xiang-Gu)
pkg/sql/catalog/rewrite/rewrite.go line 397 at r3 (raw file):
} // SchemaChangerStateInDescs goes over all mutable descriptors and cleans any
Can you expand this commentary to make it more clear what this does? I feel like the name could also be clearer. How about MaybeClearSchemaChangerStateInDescs? Explain why it exists and how it ought to be used.
Code quote:
// SchemaChangerStateInDescs goes over all mutable descriptors and cleans any
// empty state information.
func SchemaChangerStateInDescs(descriptors []catalog.MutableDescriptor) error {pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_add_column.go line 170 at r3 (raw file):
if desc.HasDefault() { expression := b.WrapExpression(tbl.TableID, cdd.DefaultExpr) // Sequence references inside expressions are unsupported, since these will
Is this no longer true?
pkg/sql/schemachanger/testdata/alter_table_add_column line 15 at r3 (raw file):
test ALTER TABLE db.public.tbl ADD COLUMN j INT NOT NULL DEFAULT nextval('db.public.sq1') ----
nit: add a separate test
f9bd08e to
babf830
Compare
fqazi
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (waiting on @ajwerner, @dt, and @Xiang-Gu)
pkg/sql/catalog/rewrite/rewrite.go line 397 at r3 (raw file):
Previously, ajwerner wrote…
Can you expand this commentary to make it more clear what this does? I feel like the name could also be clearer. How about
MaybeClearSchemaChangerStateInDescs? Explain why it exists and how it ought to be used.
// MaybeClearSchemaChangerStateInDescs goes over all mutable descriptors and
// cleans any state information from descriptors which have no targets associated
// with the corresponding jobs. Not having any targets in an individual state
// is not enough, since the descriptor in question could be a dependent (for
// example a type reference in a wider schema change).
pkg/sql/schemachanger/scbuild/internal/scbuildstmt/alter_table_add_column.go line 170 at r3 (raw file):
Previously, ajwerner wrote…
Is this no longer true?
Yes, this limitation is now lifted, we allowed sequence expressions inside default expressions.
pkg/sql/schemachanger/testdata/alter_table_add_column line 15 at r3 (raw file):
Previously, ajwerner wrote…
nit: add a separate test
Done
babf830 to
760f93a
Compare
ajwerner
left a comment
There was a problem hiding this comment.
Reviewed 1 of 4 files at r2, 6 of 9 files at r3, 3 of 3 files at r4, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @dt, @fqazi, and @Xiang-Gu)
pkg/sql/schemachanger/testdata/alter_table_add_column line 2049 at r4 (raw file):
- i - name: crdb_internal_index_7_name_placeholder + name: tbl_pkey
Should this still be called tbl_pkey here?
Fixes: cockroachdb#81781 Previously, the declarative schema changer was disabled for add column expressions with sequence references (i.e. default, on update, computed) because we were missing telemetry from the legacy schema changer for during backfill related failures. This was inadequate because the lack of telemetry from these areas could be seen as a usability issue. To address this, this patch adds in support for missing telemetry and enables supports for add column operations with sequence operations. Release note: None
760f93a to
9a9fe00
Compare
fqazi
left a comment
There was a problem hiding this comment.
Reviewable status:
complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @ajwerner, @dt, and @Xiang-Gu)
pkg/sql/schemachanger/testdata/alter_table_add_column line 2049 at r4 (raw file):
Previously, ajwerner wrote…
Should this still be called
tbl_pkeyhere?
That does look fishy, the existing rules have this issue. Let me handle this in the other PR where I'm adding UNIQUE support.
|
@ajwerner TFTR! |
|
bors r+ |
|
Build failed (retrying...): |
|
Build failed: |
|
bors r+ |
|
Build failed: |
|
bors r+ |
|
Build succeeded: |
Fixes: #81781
Previously, the declarative schema changer was disabled for
add column expressions with sequence references (i.e. default,
on update, computed) because we were missing telemetry
from the legacy schema changer for during backfill related failures.
This was inadequate because the lack of telemetry from these
areas could be seen as a usability issue. To address this,
this patch adds in support for missing telemetry and enables
supports for add column operations with sequence operations.
Release note: None