Skip to content

changefeedccl: Re-enable schema changes with CDC expressions#94653

Merged
miretskiy merged 1 commit intocockroachdb:masterfrom
miretskiy:nostop
Jan 12, 2023
Merged

changefeedccl: Re-enable schema changes with CDC expressions#94653
miretskiy merged 1 commit intocockroachdb:masterfrom
miretskiy:nostop

Conversation

@miretskiy
Copy link
Copy Markdown
Contributor

Stop requiring schema_change_policy=stop when running
changefeeds with predicates.

Fixes #84767
Epic: CRDB-17161

Release note (enterprise change): Changefeed transformations
(CREATE CHANGEFEED ... AS SELECT ...) no longer require
schema_change_policy=stop option.

@miretskiy miretskiy requested review from a team as code owners January 3, 2023 19:55
@miretskiy miretskiy requested a review from a team January 3, 2023 19:55
@miretskiy miretskiy requested a review from a team as a code owner January 3, 2023 19:55
@miretskiy miretskiy removed request for a team January 3, 2023 19:55
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@miretskiy
Copy link
Copy Markdown
Contributor Author

@HonoreDB @yuzefovich -- only the top PR needs to be reviewed.
@yuzefovich -- your input needed because of the changes to planning context(usePlannerDescriptorsForLocalFlow bit)

Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DistSQL changes LGTM, will defer to Aaron for approval.

Reviewed 8 of 11 files at r6, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB and @miretskiy)


pkg/sql/distsql_plan_changefeed.go line 198 at r6 (raw file):

	cdcPlan.PlanCtx.usePlannerDescriptorsForLocalFlow = true
	p := cdcPlan.PlanCtx.planner
	releaseDescriptors := func() {

nit: maybe s/releaseDescriptors/finishedSetupFn/?


pkg/ccl/changefeedccl/changefeed_test.go line 7052 at r6 (raw file):

		},
		{
			// Alter and rename a column  The changefeed expression does not

nit: missing period here and below.

@miretskiy
Copy link
Copy Markdown
Contributor Author

DistSQL changes LGTM, will defer to Aaron for approval.

Reviewed 8 of 11 files at r6, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @HonoreDB and @miretskiy)

pkg/sql/distsql_plan_changefeed.go line 198 at r6 (raw file):

	cdcPlan.PlanCtx.usePlannerDescriptorsForLocalFlow = true
	p := cdcPlan.PlanCtx.planner
	releaseDescriptors := func() {

nit: maybe s/releaseDescriptors/finishedSetupFn/?

pkg/ccl/changefeedccl/changefeed_test.go line 7052 at r6 (raw file):

		},
		{
			// Alter and rename a column  The changefeed expression does not

nit: missing period here and below.

all done.

@miretskiy miretskiy force-pushed the nostop branch 2 times, most recently from bc8e7d8 to ce4d48d Compare January 4, 2023 13:16
@miretskiy
Copy link
Copy Markdown
Contributor Author

@HonoreDB -- can you take a look?

@miretskiy miretskiy force-pushed the nostop branch 2 times, most recently from 5e9a109 to da12696 Compare January 5, 2023 23:34
Copy link
Copy Markdown
Contributor

@HonoreDB HonoreDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 15 files at r1, 44 of 44 files at r14, 16 of 16 files at r15, 22 of 22 files at r16, 21 of 21 files at r17, 16 of 16 files at r18, 7 of 7 files at r19, 14 of 16 files at r20.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @miretskiy and @yuzefovich)


pkg/ccl/changefeedccl/changefeedbase/options.go line 952 at r20 (raw file):

	if s.m[OptFormat] == string(OptFormatParquet) {
		if isPredicateChangefeed {
			// Diff option is allowed when using predicate changefeeds with parquet format.

Why is parquet different?

Stop requiring `schema_change_policy=stop` when running
changefeeds with predicates.

Fixes cockroachdb#84767
Epic: CRDB-17161

Release note (enterprise change): Changefeed transformations
(`CREATE CHANGEFEED ... AS SELECT ...`) no longer require
`schema_change_policy=stop` option.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cdc: handle column renames consistently in cdc expressions

4 participants