parquet: unwrap datums better#112693
Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom Oct 19, 2023
Merged
Conversation
Member
|
This change is |
6b73b98 to
81d6bdc
Compare
Previously, we did not unwrap datums read from parquet files before comparing. Now, we do. Before cockroachdb#111906, we didn't need to, but now we do because that patch added a new decoder which may return an OID-wrapped datum. We do not need to preserve OID wrappers etc when reading parquet files in tests because we do not store that information in the parquet files. Testing that OID wrappers match before and after is unecessary. Additionally, we used to skip unwrapping `REFCURSOR` datums before comparing. For the same reason as above, we can just unwrap them. This change also fixes the `validateDatum` function in exporter tests. It previously did not unwrap array contents. Now it does. Release note: None Epic: None Closes: cockroachdb#112561 Please enter the commit message for your changes. Lines starting
81d6bdc to
763f905
Compare
DrewKimball
approved these changes
Oct 19, 2023
Collaborator
DrewKimball
left a comment
There was a problem hiding this comment.
Reviewed 2 of 2 files at r1, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @miretskiy and @yuzefovich)
Contributor
Author
|
bors r+ |
miretskiy
approved these changes
Oct 19, 2023
Contributor
Author
|
bors retry |
Contributor
|
Already running a review |
Contributor
|
Build succeeded: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, we did not unwrap datums read from parquet files before
comparing. Now, we do. Before #111906, we didn't need to, but now we
do because that patch added a new decoder which may return an
OID-wrapped datum.
We do not need to preserve OID wrappers etc when reading parquet
files in tests because we do not store that information in the parquet
files. Testing that OID wrappers match before and after is unecessary.
Additionally, we used to skip unwrapping
REFCURSORdatums beforecomparing. For the same reason as above, we can just unwrap them.
This change also fixes the
validateDatumfunction in exportertests. It previously did not unwrap array contents. Now it does.
Release note: None
Epic: None
Closes: #112561