Skip to content

[SPARK-9876] Revert "[SPARK-9876][SQL] Update Parquet to 1.8.1."#13445

Closed
yhuai wants to merge 1 commit into
apache:masterfrom
yhuai:revertParquet1.8.1
Closed

[SPARK-9876] Revert "[SPARK-9876][SQL] Update Parquet to 1.8.1."#13445
yhuai wants to merge 1 commit into
apache:masterfrom
yhuai:revertParquet1.8.1

Conversation

@yhuai

@yhuai yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Since we are pretty late in the 2.0 release cycle, it is not clear if this upgrade can be tested thoroughly and if we can resolve the regression issue that we observed before. This PR temporarily reverts #13280. Let's revisit it after the release of 2.0.

How was this patch tested?

Existing tests

This reverts commit 776d183.

@rdblue

rdblue commented Jun 1, 2016

Copy link
Copy Markdown
Contributor

Sounds reasonable, but the "regression" wasn't located or even confirmed to exist after this change was reverted the last time. There was also no follow-up on it. If we revert the change I suggest that we apply it to master as soon as 2.0 is branched. That way we don't adversely affect 2.0, but we do get this addressed. Is that a reasonable path forward?

@yhuai

yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor Author

@rdblue Since 2.0 branch has been cut, I am little bit concerned about potential merge conflicts when we cherry-pick bug fixes into 2.0 branch before the release if we do not revert it from the master. We can revisit the upgrade after the release and resolve the regression issue (We will run our benchmark again to check this issue)

@yhuai

yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor Author

btw, we observed an error when filter pushdown is enabled. Unfortunately, we missed the exception...

@SparkQA

SparkQA commented Jun 1, 2016

Copy link
Copy Markdown

Test build #59746 has finished for PR 13445 at commit 82812f5.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yhuai

yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor Author

@rdblue How about we merge this to master and branch 2.0? Feel free to open your PR again. We can figure out the perf thing with @liancheng together.

@yhuai

yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor Author

OK. Let me create another PR for branch 2.0. We will merge that one first.

@yhuai

yhuai commented Jun 1, 2016

Copy link
Copy Markdown
Contributor Author

I am closing this for now.

@yhuai yhuai closed this Jun 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants