Skip to content

sql/opt: add implicit SELECT FOR UPDATE support for UPSERT statements#53132

Merged
craig[bot] merged 3 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/upsertImplicitSFU
Aug 20, 2020
Merged

sql/opt: add implicit SELECT FOR UPDATE support for UPSERT statements#53132
craig[bot] merged 3 commits intocockroachdb:masterfrom
nvb:nvanbenschoten/upsertImplicitSFU

Conversation

@nvb
Copy link
Copy Markdown
Contributor

@nvb nvb commented Aug 20, 2020

Fixes #50180.

This commit adds support for implicit SELECT FOR UPDATE support for UPSERT statements with a VALUES clause. This should improve throughput and latency for contended UPSERT statements in much the same way that 435fa43 did for UPDATE statements. However, this only has an effect on UPSERT statements into tables with multiple indexes because UPSERT statements into single-index tables hit a fast-path where they perform a blind-write without doing an initial row scan.

Conceptually, if we picture an UPSERT statement as the composition of a SELECT statement and an INSERT statement (with loosened semantics around existing rows) then this change performs the following transformation:

UPSERT t = SELECT FROM t + INSERT INTO t
=>
UPSERT t = SELECT FROM t FOR UPDATE + INSERT INTO t

I plan to test this out on a contended indexes workload at some point in the future.

Release note (sql change): UPSERT statements now acquire locks using the FOR UPDATE locking mode during their initial row scan, which improves performance for contended workloads when UPSERTing into tables with multiple indexes. This behavior is configurable using the enable_implicit_select_for_update session variable and the sql.defaults.implicit_select_for_update.enabled cluster setting.

@nvb nvb requested a review from a team as a code owner August 20, 2020 16:28
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained

nvb added 3 commits August 20, 2020 13:45
Fixes cockroachdb#50180.

This commit adds support for implicit SELECT FOR UPDATE support for
UPSERT statements with a VALUES clause. This should improve throughput
and latency for contended UPSERT statements in much the same way that
435fa43 did so for UPDATE statements. However, this only has an effect on
UPSERT statements into tables with multiple indexes because UPSERT
statements into single-index tables hit a fast-path where they perform a
blind-write without doing an initial row scan.

Conceptually, if we picture an UPSERT statement as the composition of a
SELECT statement and an INSERT statement (with loosened semantics around
existing rows) then this change performs the following transformation:
```
UPSERT t = SELECT FROM t + INSERT INTO t
=>
UPSERT t = SELECT FROM t FOR UPDATE + INSERT INTO t
```

I plan to test this out on a contended `indexes` workload at some point
in the future.

Release note (sql change): UPSERT statements now acquire locks using the
FOR UPDATE locking mode during their initial row scan, which improves
performance for contended workloads. This behavior is configurable using
the enable_implicit_select_for_update session variable and the
sql.defaults.implicit_select_for_update.enabled cluster setting.
Reverts a portion of cockroachdb#53003. These attributes are not included when they
are not interesting, but when they are included, they are very interesting
and deserve to be surfaced.
Controls the number of keys repeatedly accessed by each
writer through upserts. Mirrors the same flag in `kv`.
@nvb nvb force-pushed the nvanbenschoten/upsertImplicitSFU branch from 0292c4d to d67890c Compare August 20, 2020 17:46
@nvb nvb changed the title [DNM] sql/opt: add implicit SELECT FOR UPDATE support for UPSERT statements sql/opt: add implicit SELECT FOR UPDATE support for UPSERT statements Aug 20, 2020
@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Aug 20, 2020

I plan to test this out on a contended indexes workload at some point in the future.

To test this, I ran a quick, unscientific experiment on my laptop. Using the indexes workload's new --cycle-length flag, I created a setup with a decent amount of contention. --concurrency was set to 32 and --cycle-length was set to 32. I then varied the number of secondary indexes while switching implicit SELECT FOR UPDATE on and off.

throughput

latency

For --secondary-indexes=0, implicit SFU made no difference because of the fast-path discussed above. For all other cases, we see the same trends that we saw with implicit SFU for UPDATE statements. First, we see that throughput is a little over 50% higher using implicit SFU. Second, we see that tail latency is about 90% lower using implicit SFU.

Again, this was an unscientific experiment, but it makes a pretty strong claim that this change is beneficial for contended UPSERT statements.

@nvb
Copy link
Copy Markdown
Contributor Author

nvb commented Aug 20, 2020

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 20, 2020

Build succeeded:

@craig craig bot merged commit 979127c into cockroachdb:master Aug 20, 2020
@nvb nvb deleted the nvanbenschoten/upsertImplicitSFU branch August 21, 2020 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql/opt: add implicit SELECT FOR UPDATE support for UPSERT statements

3 participants