Skip to content

sql: implement REPLACE FUNCTION#85900

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
chengxiong-ruan:udf-replace-function
Aug 16, 2022
Merged

sql: implement REPLACE FUNCTION#85900
craig[bot] merged 1 commit intocockroachdb:masterfrom
chengxiong-ruan:udf-replace-function

Conversation

@chengxiong-ruan
Copy link
Copy Markdown
Contributor

@chengxiong-ruan chengxiong-ruan commented Aug 10, 2022

This commit adds to the REPLACE path to the
CREATE OR REPLACE FUNCTION statement. Major changes are
(1) fetch function with same signature if exists, and validate
(2) remove refereces before replacing the function, and then
add new references.

Fixes #83236

Release note: None
Release justification: This commit includes a fix to avoid duplicate functions.

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@chengxiong-ruan chengxiong-ruan force-pushed the udf-replace-function branch 4 times, most recently from 7e578e0 to 64f3051 Compare August 11, 2022 17:39
@chengxiong-ruan chengxiong-ruan marked this pull request as ready for review August 11, 2022 22:43
@chengxiong-ruan chengxiong-ruan requested review from a team and ajwerner August 11, 2022 22:43
@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

pkg/sql/create_function.go line 96 at r1 (raw file):

		}
	}
	if udfDesc.LeakProof && udfDesc.Volatility != catpb.Function_IMMUTABLE {

I'll move this into (and the one in replaceFunction the new validation helper added here : #85922

Code quote:

	if udfDesc.LeakProof && udfDesc.Volatility != catpb.Function_IMMUTABLE {
		return pgerror.Newf(
			pgcode.InvalidFunctionDefinition,
			"cannot create leakproof function with non-immutable volatility: %s",
			udfDesc.Volatility.String(),
		)
	}

Copy link
Copy Markdown
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: mod the nits

Reviewed 2 of 6 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @chengxiong-ruan)


pkg/sql/crdb_internal.go line 2594 at r1 (raw file):

`,
	populate: func(ctx context.Context, p *planner, db catalog.DatabaseDescriptor, addRow func(...tree.Datum) error) error {
		flags := tree.ObjectLookupFlags{CommonLookupFlags: tree.CommonLookupFlags{AvoidLeased: true}}

I think this is only reasonable to do because of the work @jasonmchan did such that by the time we're here, we have every descriptor 🎉 . I also think it means that you'd end up using a read descriptor rather than a leased one anyway. I guess one thing that might be cool would be to add an rttyanalysis test for a query against this table.


pkg/sql/create_function.go line 195 at r1 (raw file):

		}
		udfMutableDesc.DependsOnTypes = append(udfMutableDesc.DependsOnTypes, id)
	}

Use catalog.DescIDSet and build the first set and difference the second and then call Ordered?

Code quote:

	udfDesc.DependsOnTypes = []descpb.ID{}
	for id := range n.typeDeps {
		if implicitTypeTblIDs.Contains(int(id)) {
			continue
		}
		udfDesc.DependsOnTypes = append(udfDesc.DependsOnTypes, id)

pkg/sql/create_function.go line 299 at r1 (raw file):

	// tables used directly in function body or as implicit types.
	backrefTblIDs := util.MakeFastIntSet()
	implicitTypeTblIDs := util.MakeFastIntSet()

catalog.DescIDSet?

Code quote:

	backrefTblIDs := util.MakeFastIntSet()
	implicitTypeTblIDs := util.MakeFastIntSet()

pkg/sql/create_function.go line 376 at r1 (raw file):

	// Add forward references to UDF descriptor.
	udfDesc.DependsOn = make([]descpb.ID, 0, backrefTblIDs.Len())

udfDesc.DependsOn = backrefTblIDs.Ordered()?

Code quote:

	udfDesc.DependsOn = make([]descpb.ID, 0, backrefTblIDs.Len())
	backrefTblIDs.ForEach(func(id int) {
		udfDesc.DependsOn = append(udfDesc.DependsOn, descpb.ID(id))
	})

Copy link
Copy Markdown
Contributor Author

@chengxiong-ruan chengxiong-ruan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @jasonmchan)


pkg/sql/crdb_internal.go line 2594 at r1 (raw file):

Previously, ajwerner wrote…

I think this is only reasonable to do because of the work @jasonmchan did such that by the time we're here, we have every descriptor 🎉 . I also think it means that you'd end up using a read descriptor rather than a leased one anyway. I guess one thing that might be cool would be to add an rttyanalysis test for a query against this table.

Oh yeah, that's awesome work! Though I didn't do ReadAllDescriptors() here, should I do that? Do you think it's an overkill here given the number of schema and function could be low?


pkg/sql/create_function.go line 195 at r1 (raw file):

Previously, ajwerner wrote…

Use catalog.DescIDSet and build the first set and difference the second and then call Ordered?

that saved my life, fixed, thanks!


pkg/sql/create_function.go line 299 at r1 (raw file):

Previously, ajwerner wrote…

catalog.DescIDSet?

ah, thanks, I kept forgetting this. fixed


pkg/sql/create_function.go line 376 at r1 (raw file):

Previously, ajwerner wrote…

udfDesc.DependsOn = backrefTblIDs.Ordered()?

yes, with ID set, this is much nicer. fixed

Copy link
Copy Markdown
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @ajwerner and @jasonmchan)


pkg/sql/crdb_internal.go line 2594 at r1 (raw file):

Previously, chengxiong-ruan (Chengxiong Ruan) wrote…

Oh yeah, that's awesome work! Though I didn't do ReadAllDescriptors() here, should I do that? Do you think it's an overkill here given the number of schema and function could be low?

Oh, I suppose that that's right. I don't know. It's one of those things where it's really painful if the nodes are far apart. I'm hopeful in 23.1 to adopt global tables, but for now it's going to be O(functions) round-trips which might be very slow. One approach might be to collect the IDs in one pass and then fetch them as a batch and then iterate separately.

@chengxiong-ruan chengxiong-ruan force-pushed the udf-replace-function branch 2 times, most recently from 7db9f60 to 120fe55 Compare August 15, 2022 15:32
@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

pkg/sql/crdb_internal.go line 2594 at r1 (raw file):

Previously, ajwerner wrote…

Oh, I suppose that that's right. I don't know. It's one of those things where it's really painful if the nodes are far apart. I'm hopeful in 23.1 to adopt global tables, but for now it's going to be O(functions) round-trips which might be very slow. One approach might be to collect the IDs in one pass and then fetch them as a batch and then iterate separately.

I see, thanks for the background info. And thanks for the batching idea, I changed it that way

@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

TFTR!
bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 15, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 15, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 15, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 16, 2022

Build failed:

@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

hmm bizarre timeout on some backup tests
bors r-

This commit adds to the `REPLACE` path to the
`CREATE OR REPLACE FUNCTION` statement. Major changes are
(1) fetch function with same signature if exists, and validate
(2) remove refereces before replacing the function, and then
    add new references.

Release note: None
@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

Looks like the time out was fixed overnight maybe by #86191

@ajwerner
Copy link
Copy Markdown
Contributor

I'm worried there's still a flake in there somewhere. If you see one, let me know.

@chengxiong-ruan
Copy link
Copy Markdown
Contributor Author

🙏
bors r+

@craig craig bot merged commit 7c38417 into cockroachdb:master Aug 16, 2022
@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 16, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UDF: Support CREATE OR REPLACE FUNCTION statement

3 participants