Skip to content

sql: refactor sessionDataMutator as sessionDataMutatorIterator#69394

Merged
craig[bot] merged 4 commits intocockroachdb:masterfrom
otan-cockroach:session_mutator_changes
Aug 26, 2021
Merged

sql: refactor sessionDataMutator as sessionDataMutatorIterator#69394
craig[bot] merged 4 commits intocockroachdb:masterfrom
otan-cockroach:session_mutator_changes

Conversation

@otan
Copy link
Copy Markdown
Contributor

@otan otan commented Aug 25, 2021

Part of a body of work for SET LOCAL (refs #32562)

See individual commits for details.

Release justification: high pri work

@otan otan requested review from a team, adityamaru and rafiss and removed request for a team August 25, 2021 23:51
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@otan otan mentioned this pull request Aug 26, 2021
Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru and @otan)


pkg/sql/conn_executor.go, line 1480 at r1 (raw file):

}

// sessionData returns the top SessionData in the executor's sessionDataStack.

it's worth mentioning in this comment that the purpose of doing this is so that each transacation can track a separate sessiondata, and so we can get txn-scoped session variable changes.


pkg/sql/exec_util.go, line 2428 at r2 (raw file):

// a transaction where there may be two or more SessionData elements in
// the stack)
type sessionDataMutatorTopOnlyBase struct {

heh i can see why this is the name, but at the same time i feel like there could be a better name

something like sessionDataMutatorTopCallbacks or even just sessionDataMutatorCallbacks?


pkg/sql/exec_util.go, line 2453 at r2 (raw file):

		sessionDataMutatorBase: f.sessionDataMutatorBase,
	}
	// Change the sessionDataMutatorTopOnlyBase if our SessionData element is not

hmm could you explain a bit more about why we need this? i get that we're going to be creating a mutator for each element in the stack, but then why change this? is this the step that prevents us from sending out multiple param status updates?

also, what is sessionDataMutatorTopOnlyBase going to be if we don't change it? it looks like it would just be nil?

(i'm just asking since i couldn't piece it together on my own. but maybe this info would be useful to put in this comment.)


pkg/sql/exec_util.go, line 2455 at r2 (raw file):

	// Change the sessionDataMutatorTopOnlyBase if our SessionData element is not
	// the top.
	if sd != f.sds.Top() {

comparing the pointers seems a bit fragile to me.. but maybe my instincts are wrong on this. what would you feel about making each sessiondata keep track of its nesting level, and comparing those instead?


pkg/sql/exec_util.go, line 2486 at r2 (raw file):

) error {
	for _, sd := range f.sds.Elems() {
		if err := applyFunc(f.mutator(sd)); err != nil {

could we end up in a bad state if we error out of this loop early? it would mean that some of the sessiondatas had the function applied and some did not. e.g. in the case of RESET ALL, some levels of the txn nesting would get reset while others would not

the ideal would be that if we encounter an error, we undo all the changes that were done so far. idk if it's feasible to do so -- maybe we could copy each sessiondata before calling applyFunc on it, and revert the stack elements to those copies?


pkg/sql/exec_util.go, line 2493 at r2 (raw file):

}

// sessionDataMutator is the object used by sessionVars to change the session

should sessionVars still be using this? why don't they need to use the sessionDataMutatorIterator?

actually, why do we need this struct at all? can't we get rid of this sessionDataMutator and rename sessionDataMutatorIterator to sessionDataMutator?


pkg/sql/sessiondata/session_data.go, line 206 at r1 (raw file):

func (s *Stack) Top() *SessionData {
	if len(s.stack) == 0 {
		return nil

another followup from the last PR:

should this function return an AssertionError here if the stack is empty?

Copy link
Copy Markdown
Contributor Author

@otan otan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru and @rafiss)


pkg/sql/conn_executor.go, line 1480 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

it's worth mentioning in this comment that the purpose of doing this is so that each transacation can track a separate sessiondata, and so we can get txn-scoped session variable changes.

Yep.


pkg/sql/exec_util.go, line 2428 at r2 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

heh i can see why this is the name, but at the same time i feel like there could be a better name

something like sessionDataMutatorTopCallbacks or even just sessionDataMutatorCallbacks?

Done.


pkg/sql/exec_util.go, line 2453 at r2 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

hmm could you explain a bit more about why we need this? i get that we're going to be creating a mutator for each element in the stack, but then why change this? is this the step that prevents us from sending out multiple param status updates?

also, what is sessionDataMutatorTopOnlyBase going to be if we don't change it? it looks like it would just be nil?

(i'm just asking since i couldn't piece it together on my own. but maybe this info would be useful to put in this comment.)

it prevents us from sending multiple paramstatusupdates for a SET x inside a transaction. otherwise, it would create paramstatusupdate for each level of nesting in the txn!


pkg/sql/exec_util.go, line 2455 at r2 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

comparing the pointers seems a bit fragile to me.. but maybe my instincts are wrong on this. what would you feel about making each sessiondata keep track of its nesting level, and comparing those instead?

done!


pkg/sql/exec_util.go, line 2486 at r2 (raw file):

could we end up in a bad state if we error out of this loop early?

potentially, but I'm not sure it's worth doing much more than that until we do #69391. in theory, if something errors here, we're in bigger trouble.

seems like the savepointStack has a similar issue if we're paranoid, but I'm ok with things as is.


pkg/sql/exec_util.go, line 2493 at r2 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

should sessionVars still be using this? why don't they need to use the sessionDataMutatorIterator?

actually, why do we need this struct at all? can't we get rid of this sessionDataMutator and rename sessionDataMutatorIterator to sessionDataMutator?

because the set_vars.go will be responsible for delegating to either the top mutator, or all mutators.
the iterator makes it easy to transform mutator objects based on whether or not it's the top. without the separation also, we'd have to take in a islocal argument to all the varGen variables which is hairier imo.


pkg/sql/sessiondata/session_data.go, line 206 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

another followup from the last PR:

should this function return an AssertionError here if the stack is empty?

Not everything errors here; we can panic if you want. I'm ok with the other assertions we already made in the pop functions.

@otan otan force-pushed the session_mutator_changes branch from a493f86 to b2bd77f Compare August 26, 2021 05:24
@otan
Copy link
Copy Markdown
Contributor Author

otan commented Aug 26, 2021


pkg/sql/exec_util.go, line 2486 at r2 (raw file):

Previously, otan (Oliver Tan) wrote…

could we end up in a bad state if we error out of this loop early?

potentially, but I'm not sure it's worth doing much more than that until we do #69391. in theory, if something errors here, we're in bigger trouble.

seems like the savepointStack has a similar issue if we're paranoid, but I'm ok with things as is.

Actually we always reset to the first stack entry after a txn so if it's corrupt it's inside an already broken txn. So I think re ok

Copy link
Copy Markdown
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! just some optional comments if you'd like

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru and @otan)


pkg/sql/exec_util.go, line 2453 at r2 (raw file):

Previously, otan (Oliver Tan) wrote…

it prevents us from sending multiple paramstatusupdates for a SET x inside a transaction. otherwise, it would create paramstatusupdate for each level of nesting in the txn!

it's more clear to me now thanks!


pkg/sql/exec_util.go, line 2486 at r2 (raw file):

Previously, otan (Oliver Tan) wrote…

Actually we always reset to the first stack entry after a txn so if it's corrupt it's inside an already broken txn. So I think re ok

excellent!


pkg/sql/exec_util.go, line 2493 at r2 (raw file):

Previously, otan (Oliver Tan) wrote…

because the set_vars.go will be responsible for delegating to either the top mutator, or all mutators.
the iterator makes it easy to transform mutator objects based on whether or not it's the top. without the separation also, we'd have to take in a islocal argument to all the varGen variables which is hairier imo.

ohh i see, i missed the change where applyFunc was added to func (n *setVarNode) startExec(). yes that definitely seems nicer than having to pass in isLocal to everything


pkg/sql/exec_util.go, line 2403 at r6 (raw file):

	defaults SessionDefaults
	settings *cluster.Settings
	sessionDataMutatorCallbacks

if we embedded sessionDataMutatorCallbacks inside of sessionDataMutatorIterator and sessionDataMutator instead of embedding it here, would it make things cleaner? then we could avoid the logic of having to unset it inside of mutator() right?


pkg/sql/exec_util.go, line 2475 at r6 (raw file):

	elems := f.sds.Elems()
	for i, sd := range elems {
		applyFunc(f.mutator(i == len(elems)-1, sd))

we just as well could change this to i == 0 -- that way the param status notifications are sent as soon as the value is updated in the top-level, outside of any txn. and if any of the per-txn data mutators fail, the param status will still have been sent out. (ditto for forEachMutatorError)

Copy link
Copy Markdown
Contributor Author

@otan otan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @adityamaru, @otan, and @rafiss)


pkg/sql/exec_util.go, line 2493 at r2 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

ohh i see, i missed the change where applyFunc was added to func (n *setVarNode) startExec(). yes that definitely seems nicer than having to pass in isLocal to everything

Done.


pkg/sql/exec_util.go, line 2403 at r6 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

if we embedded sessionDataMutatorCallbacks inside of sessionDataMutatorIterator and sessionDataMutator instead of embedding it here, would it make things cleaner? then we could avoid the logic of having to unset it inside of mutator() right?

good idea, done!


pkg/sql/exec_util.go, line 2475 at r6 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

we just as well could change this to i == 0 -- that way the param status notifications are sent as soon as the value is updated in the top-level, outside of any txn. and if any of the per-txn data mutators fail, the param status will still have been sent out. (ditto for forEachMutatorError)

Done.

@otan
Copy link
Copy Markdown
Contributor Author

otan commented Aug 26, 2021

bors r=rafiss

thanks!

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 26, 2021

Merge conflict.

otan added 3 commits August 26, 2021 18:00
Leftover comment fixes from an earlier PR.

* clear pointers from the array before slicing down
* fixed up a few comments

Release justification: comment only changes / low risk high pri bug
fixes

Release note: None
This commit converts existing usages of sessionDataMutator to
sessionDataMutatorIterator so that when we introduce SET LOCAL, we are
explicitly either setting *every* sessiondata in the stack for SET
SESSION or just the "top" element for SET LOCAL.

sessionDataMutatorIterator contains a sessionDataMutatorBase element
which is used to create a sessionDataMutator, where the remaining
usages can be used as is throughout the codebase. Most of the work here
is to change the `mutator.Set*` objects to be set through
`forEachMutator` instead.

Release justification: high priority feature request

Release note: None
Instead of adding this noop interface, only trigger the param status
updater if it is not nil in sessionDataMutator. This saves a lot of
bootstrapping work.

Release justification: small refactor change

Release note: None
@otan otan force-pushed the session_mutator_changes branch from 2f0f79d to 8315580 Compare August 26, 2021 08:02
@otan
Copy link
Copy Markdown
Contributor Author

otan commented Aug 26, 2021

bors r=rafiss

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 26, 2021

Build failed:

Move sessionDataMutatorCallbacks out of the base and onto the iterator
and the mutator itself. This means we no longer have to do the reassign
business.

Release justification: urgent new functionality
Release note: None
@otan otan force-pushed the session_mutator_changes branch from 8315580 to d7d05b2 Compare August 26, 2021 09:08
@otan
Copy link
Copy Markdown
Contributor Author

otan commented Aug 26, 2021

bors r=rafiss

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 26, 2021

Build succeeded:

@craig craig bot merged commit 2e949fa into cockroachdb:master Aug 26, 2021
craig bot pushed a commit that referenced this pull request Aug 27, 2021
69224: sql: implement SET LOCAL r=rafiss a=otan

Resolves: #32562
Release justification: high priority bug fix

First 3 commits from #69394

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
craig bot pushed a commit that referenced this pull request Aug 27, 2021
69224: sql: implement SET LOCAL r=rafiss a=otan

Resolves: #32562
Release justification: high priority bug fix

First 3 commits from #69394

Co-authored-by: Oliver Tan <otan@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants