Skip to content

opt: fix bug with incorrect results produced by paired left lookup join#82054

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rytaft:paired-join-bug
May 30, 2022
Merged

opt: fix bug with incorrect results produced by paired left lookup join#82054
craig[bot] merged 1 commit intocockroachdb:masterfrom
rytaft:paired-join-bug

Conversation

@rytaft
Copy link
Copy Markdown
Collaborator

@rytaft rytaft commented May 30, 2022

Prior to this patch, it was possible for a paired join to produce incorrect
results for a left lookup join. In particular, some output rows had
non-NULL values for right-side columns when the right-side columns should
have been NULL.

This commit fixes the issue by updating the optimizer to ensure that
only columns from the second join in the paired join (the index join)
are projected, not columns from the first (the lookup join).

Fixes #81968

Release note (bug fix): Fixed an issue where a left lookup join could
have incorrect results. In particular, some output rows could have non-NULL
values for right-side columns when the right-side columns should
have been NULL. This issue only exists in 22.1.0 and prior development
releases of 22.1.

@rytaft rytaft requested a review from a team as a code owner May 30, 2022 01:53
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

Copy link
Copy Markdown
Contributor

@msirek msirek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The column remapping looks complete and I could find no other variables based on scanPrivate, lookupJoin, or lookupConstraint that need an update (such as was done with pkCols).

:lgtm:

Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @mgartner, @sumeerbhola, and @yuzefovich)

Copy link
Copy Markdown
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix! LGTM too.

Reviewed 6 of 6 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @mgartner, @rytaft, and @sumeerbhola)


pkg/sql/logictest/testdata/logic_test/lookup_join line 790 at r1 (raw file):

    chat_id   INT NOT NULL,
    author_id INT NOT NULL,
    INDEX chat_id_idx (chat_id) -- without this index everything works fine

nit: not sure if we want to keep this comment, up to you.

Prior to this patch, it was possible for a paired join to produce incorrect
results for a left lookup join. In particular, some output rows had
non-NULL values for right-side columns when the right-side columns should
have been NULL.

This commit fixes the issue by updating the optimizer to ensure that
only columns from the second join in the paired join (the index join)
are projected, not columns from the first (the lookup join).

Fixes cockroachdb#81968

Release note (bug fix): Fixed an issue where a left lookup join could
have incorrect results. In particular, some output rows could have non-NULL
values for right-side columns when the right-side columns should
have been NULL. This issue only exists in 22.1.0 and prior development
releases of 22.1.
@rytaft rytaft force-pushed the paired-join-bug branch from 9145f2a to 0cb0dca Compare May 30, 2022 12:57
Copy link
Copy Markdown
Collaborator Author

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner, @msirek, @sumeerbhola, and @yuzefovich)


pkg/sql/logictest/testdata/logic_test/lookup_join line 790 at r1 (raw file):

Previously, yuzefovich (Yahor Yuzefovich) wrote…

nit: not sure if we want to keep this comment, up to you.

Done.

@craig
Copy link
Copy Markdown
Contributor

craig bot commented May 30, 2022

Build succeeded:

@craig craig bot merged commit cf8fddc into cockroachdb:master May 30, 2022
@blathers-crl
Copy link
Copy Markdown

blathers-crl bot commented May 30, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 0cb0dca to blathers/backport-release-22.1-82054: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

@mgartner
Copy link
Copy Markdown
Contributor

Great find and fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

opt: left lookup join pair does not correctly null-extend unmatched rows

5 participants