importccl: lift computed column check to input converter#55845
Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom Oct 27, 2020
Merged
importccl: lift computed column check to input converter#55845craig[bot] merged 2 commits intocockroachdb:masterfrom
craig[bot] merged 2 commits intocockroachdb:masterfrom
Conversation
Member
Contributor
|
That was fast! I'm curious if you see a way to get rid of https://github.com/cockroachdb/cockroach/pull/55845/files#diff-e56c3f528820bd6c95bc632c127d4ef1b5ae752a8accb667690a5e07c7c9b25eR137 as well. What are the cases where |
Contributor
Author
|
We expect that case when there are no "targetted" columns. In #55846 is built on top of this, and I think that should do the trick. I took a couple of quick profiles and it looks like that change got rid of VisibleColumns from the Row() profile as expected. |
92d82a2 to
555457d
Compare
555457d to
007bc36
Compare
dt
approved these changes
Oct 26, 2020
The current implementation of checking for validating the number of rows for computed columns for in non-IMPORT INTO backups is inefficient. This commit moves the check from being performed on every row to only being performed once per import. Release note: None
007bc36 to
caaf981
Compare
This commit moves the computation of finding the expected number of data columns to the creation of the input converter, rather than re-calculating it on every row. The memory footprint of loading all of the visible columns for every row was noticeable. Release note (performance improvement): CSV imports should now be slightly faster.
caaf981 to
6a9eaf3
Compare
Contributor
Author
|
TFTR! |
Contributor
|
Build succeeded: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The current implementation of checking for validating the number of rows
for computed columns for in non-IMPORT INTO backups is inefficient. This
commit moves the check from being performed on every row to only being
performed once per import.
Fixes #55842.
Release note: None