Skip to content

opt: match ScalarGroupBy in EliminateIndexJoinOrProjectInsideGroupBy#70120

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
mgartner:70116-partial-index-validation
Sep 14, 2021
Merged

opt: match ScalarGroupBy in EliminateIndexJoinOrProjectInsideGroupBy#70120
craig[bot] merged 1 commit intocockroachdb:masterfrom
mgartner:70116-partial-index-validation

Conversation

@mgartner
Copy link
Copy Markdown
Contributor

This commit extends EliminateIndexJoinOrProjectInsideGroupBy so that
ScalarGroupBy expressions are also matched. This allows the rule to
eliminate unnecessary index joins in more cases.

The primary motivation for this change was to make partial index
validation queries more efficient. These queries always have
ScalarGroupBy expressions because they are in the form:

SELECT count(1) FROM table@partial_index WHERE predicate

Prior to this change, an index join was planned for these queries which
would operate on every row in the partial index. This could be extremely
expensive for large partial indexes.

Fixes #70116

Release note (performance improvement): A limitation has been fixed that
made creating partial indexes inefficient.

@mgartner mgartner requested review from a team, RaduBerinde and rytaft September 13, 2021 13:57
@mgartner mgartner requested a review from a team as a code owner September 13, 2021 13:57
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

This commit extends EliminateIndexJoinOrProjectInsideGroupBy so that
ScalarGroupBy expressions are also matched. This allows the rule to
eliminate unnecessary index joins in more cases.

The primary motivation for this change was to make partial index
validation queries more efficient. These queries always have
ScalarGroupBy expressions because they are in the form:

    SELECT count(1) FROM table@partial_index WHERE predicate

Prior to this change, an index join was planned for these queries which
would operate on every row in the partial index. This could be extremely
expensive for large partial indexes.

Fixes cockroachdb#70116

Release note (performance improvement): A limitation has been fixed that
made creating partial indexes inefficient.
@mgartner mgartner force-pushed the 70116-partial-index-validation branch from aabc3e8 to 2751851 Compare September 13, 2021 14:53
Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rytaft)

Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a partial index validation query used for?

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rytaft)

Copy link
Copy Markdown
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, never mind, we do it just to double-check the backfill.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @rytaft)

@mgartner
Copy link
Copy Markdown
Contributor Author

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Sep 14, 2021

Build succeeded:

@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented Sep 14, 2021

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 2751851 to blathers/backport-release-21.1-70120: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 21.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

@mgartner
Copy link
Copy Markdown
Contributor Author

Backporting this to 21.1 would require also backporting 64bb06a. Given that there is a workaround for this (STORING the columns referenced in the predicate), I'm inclined to not backport to 21.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sql: remove unnecessary index joins in partial index validation queries

3 participants