Some minor cleanup#4
Merged
SinghAsDev merged 1 commit intoSinghAsDev:PARQUET-251from Jun 30, 2015
isnotinvain:PR-197-3
Merged
Conversation
SinghAsDev
pushed a commit
that referenced
this pull request
Jul 2, 2015
Author: asingh <asingh@cloudera.com> Author: Alex Levenson <alexlevenson@twitter.com> Author: Ashish Singh <asingh@cloudera.com> Closes apache#197 from SinghAsDev/PARQUET-251 and squashes the following commits: 68e0eae [asingh] Remove deprecated constructors from private classes 67e4e5f [asingh] Add removed public methods in Binary and deprecate them 0e71728 [asingh] Add comment for BinaryStatistics.setMinMaxFromBytes fbe873f [Ashish Singh] Merge pull request #4 from isnotinvain/PR-197-3 9826ee6 [Alex Levenson] Some minor cleanup 7570035 [asingh] Remove test for stats getting ingnored for version 160 when type is int64 af43d28 [Alex Levenson] Address PR feedback 89ab4ee [Alex Levenson] put the headers in the right location 2838cc9 [Alex Levenson] Split out version checks to separate files, add some tests 5af9142 [Alex Levenson] Generalize tests, make Binary.fromString reused=false e00d9b7 [asingh] Rename isReused => isBackingBytesReused d2ad939 [asingh] Rebase over latest trunk 857141a [asingh] Remove redundant junit dependency 32b88ed [asingh] Remove semver from hadoop-common 7a0e99e [asingh] Revert to fromConstantByteArray for ByteString c820ec9 [asingh] Add unit tests for Binary and to check if stats are ignored for version 160 9bbd1e5 [asingh] Improve version parsing 84a1d8b [asingh] Remove ignoring stats on write side and ignore it on read side 903f8e3 [asingh] Address some review comments. * Ignore stats for writer's version < 1.8.0 * Refactor shoudlIgnoreStatistics method a bit * Assume implementations other than parquet-mr were writing binary statistics correctly * Add toParquetStatistics method's original method signature to maintain backwards compatibility and mark it as deprecated 64c2617 [asingh] Revert changes for ignoring stats at RowGroupFilter level e861b18 [asingh] Ignore max min stats while reading 3a8cb8d [asingh] Fix typo 8e12618 [asingh] Fix usage of fromConstant versions of Binary constructors 860adf7 [asingh] Rename unmodified to constant and isReused instead of isUnmodifiable 0d127a7 [asingh] Add unmodfied and Reused versions for creating a Binary. Add copy() to Binary. b4e2950 [asingh] Skip filtering based on stats when file was written with version older than 1.6.1 6fcee8c [asingh] Add getBytesUnsafe() to Binary that returns backing byte[] if possible, else returns result of getBytes() 30b07dd [asingh] PARQUET-251: Binary column statistics error when reuse byte[] among rows
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Drop some unused instance variables, add a null check for parquetMetaData