Skip to content

sql/opt/exec: perform implicit SELECT FOR UPDATE on UPDATE with index join#45733

Merged
nvb merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/implicitDelete
Mar 9, 2020
Merged

sql/opt/exec: perform implicit SELECT FOR UPDATE on UPDATE with index join#45733
nvb merged 1 commit intocockroachdb:masterfrom
nvb:nvanbenschoten/implicitDelete

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Mar 4, 2020

Fixes #45511.

This commit enables implicit row-level locking on UPDATEs with an index-join
in its initial row scan. To support this, two new patterns were needed in
Builder.shouldApplyImplicitLockingToUpdateInput:

(Update
    $input:(IndexJoin
        $input:(Scan $scanPrivate:*)
        $indexJoinPrivate:*
    )
    $checks:*
    $mutationPrivate:*
)

AND

(Update
    $input:(Project
        $input:(IndexJoin
            $input:(Scan $scanPrivate:*)
            $indexJoinPrivate:*
        )
        $projections:*
        $passthrough:*
    )
    $checks:*
    $mutationPrivate:*
)

These extra patterns are pushing the limits of this form of pattern matching,
but this is ok for the next release. If/when we want to make any more general,
we should strongly consider Andy's proposal to fold the patten matching and
analysis into the explorer rather than execbuilder.

@nvb nvb requested a review from andy-kimball March 4, 2020 23:05
@nvb nvb requested a review from a team as a code owner March 4, 2020 23:05
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@nvb nvb force-pushed the nvanbenschoten/implicitDelete branch from 15712e7 to 350b436 Compare March 4, 2020 23:37
Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball and @nvanbenschoten)


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):

		return false
	}

This is all a very complicated way of saying we want [Project] [IndexJoin] Scan (which is also an easier way to think about things). I would just peel off a Project if there is one, then peel off an IndexJoin if there is one, then check that we have a Scan.

… join

Fixes cockroachdb#45511.

This commit enables implicit row-level locking on UPDATEs with an index-join
in its initial row scan. To support this, two new patterns were needed in
`Builder.shouldApplyImplicitLockingToUpdateInput`:

```
(Update
    $input:(IndexJoin
        $input:(Scan $scanPrivate:*)
        $indexJoinPrivate:*
    )
    $checks:*
    $mutationPrivate:*
)

AND

(Update
    $input:(Project
        $input:(IndexJoin
            $input:(Scan $scanPrivate:*)
            $indexJoinPrivate:*
        )
        $projections:*
        $passthrough:*
    )
    $checks:*
    $mutationPrivate:*
)
```

These extra patterns are pushing the limits of this form of pattern matching,
but this is ok for the next release. If/when we want to make any more general,
we should strongly consider Andy's proposal to fold the patten matching and
analysis into the explorer rather than execbuilder.
@nvb nvb force-pushed the nvanbenschoten/implicitDelete branch from 350b436 to 8c000de Compare March 6, 2020 02:44
Copy link
Copy Markdown
Contributor Author

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball and @RaduBerinde)


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):

Previously, RaduBerinde wrote…

This is all a very complicated way of saying we want [Project] [IndexJoin] Scan (which is also an easier way to think about things). I would just peel off a Project if there is one, then peel off an IndexJoin if there is one, then check that we have a Scan.

That's a really good point. Done.

Copy link
Copy Markdown
Contributor Author

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball and @RaduBerinde)


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

That's a really good point. Done.

Out of curiosity, what would the corresponding optgen pattern look like for this expression pattern? Is there a way to express it succinctly?

Copy link
Copy Markdown
Contributor

@andy-kimball andy-kimball left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andy-kimball, @nvanbenschoten, and @RaduBerinde)


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Out of curiosity, what would the corresponding optgen pattern look like for this expression pattern? Is there a way to express it succinctly?

optgen doesn't have support for doing nested matching like this, so in case like this, I might write a rule like:

(Update | Delete
  $input:* & (CanApplyImplicitLocking $input)
  $checks:*
  $mutationPrivate:*
)
=>
((OpName)
  (ApplyImplicitLocking $input)
  $checks
  $mutationPrivate
)

Even though we're not getting value from optgen for the matching you need to do here, we still benefit from matching the top-level operators:

  1. It's nice to have all our patterns in the optgen language for readability.
  2. Optgen will generate multiple instances of the rule for each mutation operator it applies to, and place in right locations in factory.
  3. Optgen will generate the boilerplate code for making this work with our testing tools like optsteps and random rule suppression.

Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @nvanbenschoten and @RaduBerinde)

Copy link
Copy Markdown
Contributor Author

@nvb nvb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

Reviewable status: :shipit: complete! 2 of 0 LGTMs obtained (waiting on @andy-kimball and @RaduBerinde)


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):
Got it, thanks for expanding. The reasons for including this in optgen despite the fact that the pattern match itself will be in Go are compelling.

Optgen will generate multiple instances of the rule for each mutation operator it applies to, and place in right locations in factory.

One thing I'm not sure about is whether UPDATE and DELETE will want to match on the same patterns for their input. I think I found that they don't a few weeks ago, but I haven't returned to pulling on that thread. I'm no longer planning on pushing through implicit SFU for DELETE statements for this release because I don't know of a single workload that will benefit from them.

@nvb nvb merged commit c3baa66 into cockroachdb:master Mar 9, 2020
@andy-kimball
Copy link
Copy Markdown
Contributor


pkg/sql/opt/exec/execbuilder/mutation.go, line 804 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Got it, thanks for expanding. The reasons for including this in optgen despite the fact that the pattern match itself will be in Go are compelling.

Optgen will generate multiple instances of the rule for each mutation operator it applies to, and place in right locations in factory.

One thing I'm not sure about is whether UPDATE and DELETE will want to match on the same patterns for their input. I think I found that they don't a few weeks ago, but I haven't returned to pulling on that thread. I'm no longer planning on pushing through implicit SFU for DELETE statements for this release because I don't know of a single workload that will benefit from them.

That's fine. Often what I do in situations like this is just match on the union of patterns. In other words, even if we don't generate a Project op in the DELETE case (or whatever else), it doesn't hurt to match that pattern. It's simpler to just batch together Delete, Update, Upsert, etc.

@nvb nvb deleted the nvanbenschoten/implicitDelete branch March 16, 2020 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql/opt/exec: perform implicit SELECT FOR UPDATE on UPDATE with index join

4 participants