sql/catalog: schemachanger must not produce elements 22.1 does not have #86659
Description
Describe the problem
We need to be very careful to not produce elements in 22.2 which 22.1 does not know about. Such elements can result in panics:
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 a panic has occurred!
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 +interface conversion: interface is nil, not scpb.Element
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 +(1) attached stack trace
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + -- stack trace:
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | runtime.gopanic
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | GOROOT/src/runtime/panic.go:1038
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | runtime.assertE2I
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | GOROOT/src/runtime/iface.go:480
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/schemachanger/scpb.(*ElementProto).Element
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/schemachanger/scpb/pkg/sql/schemachanger/scpb/state.go:90
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.validateSchemaChangerState
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/schema_changer_state.go:47
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func1
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:59
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:165
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:54
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree.Catalog.Validate
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree/catalog.go:144
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*kvDescriptors).getAllDescriptors
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/kv_descriptors.go:279
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptors
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:284
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2388
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescAllWithTableLookup
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2321
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescAll
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2300
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql.glob..func73.1
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:2751
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:494
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | runtime.goexit
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 + | src/runtime/asm_amd64.s:1581
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 +Wraps: (2) interface conversion: interface is nil, not scpb.Element
E220823 13:15:02.077270 733 1@util/log/logcrash/crash_reporting.go:174 ⋮ [n1,intExec=‹capture-index-usage-stats›] 58 +Error types: (1) *withstack.withStack (2) *runtime.TypeAssertionError
To Reproduce
Run a drop job in 22.2 in the mixed version state with pausepoints. Shut down, run the code in 22.1. See the crash.
Expected behavior
The job completes happily
Jira issue: CRDB-18862
Metadata
Metadata
Assignees
Labels
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.SQL Foundations Team (formerly SQL Schema + SQL Sessions)Used to mark GA and release blockers, technical advisories, and bugs for 22.2Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked.