PARQUET-2465: Fall back to HadoopConfig (#1339)#1342
Merged
Fokko merged 1 commit intoapache:parquet-1.14.xfrom May 3, 2024
Merged
PARQUET-2465: Fall back to HadoopConfig (#1339)#1342Fokko merged 1 commit intoapache:parquet-1.14.xfrom
Fokko merged 1 commit intoapache:parquet-1.14.xfrom
Conversation
Add fallback logic
We see that this causes the 1.14 to be incompatible
with the previous releases. The config will be created
and right after that the `getWriteSupport(conf)` is called.
But since this method is freshly introduced:
```java
protected WriteSupport<T> getWriteSupport(ParquetConfiguration conf) {
throw new UnsupportedOperationException(
"Override ParquetWriter$Builder#getWriteSupport(ParquetConfiguration)");
}
```
wgtmac
approved these changes
May 3, 2024
Member
|
Thanks for backporting! BTW, I'm still on vacation until May 6th. I can restart preparation for RC1 after that if there is no hurry. |
Contributor
Author
|
@wgtmac It would be nice to get the RC out because of the Spark 4.0 release. I can also send out the mail for RC1 and then you can wrap it up when you're back from vacation |
Member
|
That sounds good. Thanks @Fokko! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backporting the fallback logic
We see that this causes the 1.14 to be incompatible with the previous releases. The config will be created and right after that the
getWriteSupport(conf)is called.But since this method is freshly introduced:
Make sure you have checked all steps below.
Jira
them in the PR title. For example, "PARQUET-1234: My Parquet PR"
the ASF 3rd Party License Policy.
Tests
Commits
from "How to write a good git commit message":
Style
mvn spotless:apply -Pvector-pluginsDocumentation