Skip to content

Conversation

@jneuff
Copy link

@jneuff jneuff commented Apr 25, 2018

The trailing space is added in parquet-cpp. pyarrow calls the function FormatStatValue which adds the trailing space (https://github.com/apache/parquet-cpp/blob/master/src/parquet/types.cc#L52).

https://issues.apache.org/jira/browse/PARQUET-1283 is about fixing this behavior. Once the corresponding PR is merged into parquet-cpp and pyarrow is synced, the test_parquet.py:test_parquet_column_statistics_api will break for str. This PR fixes that breakage.

@xhochy
Copy link
Member

xhochy commented Apr 25, 2018

I guess this depends on the PR in parquet-cpp as the statistics tests are failing?

@jneuff
Copy link
Author

jneuff commented Apr 26, 2018

Sorry, I was a bit to eager to open this PR. I want to solve it without depending on the PR in parquet-cpp.

I'll reiterate.

@jneuff jneuff force-pushed the fix-trailing-space-in-string-statistics branch from 8c54fb7 to 57cdb6f Compare April 26, 2018 12:29
@jneuff
Copy link
Author

jneuff commented Apr 26, 2018

I updated the scope of this PR, see the description above.

@pitrou
Copy link
Member

pitrou commented May 1, 2018

@jneuff, could you rebase this PR so that we get a fresh CI build?

This accounts for the fixed behavior of FormatStatValue
as described in PARQUET-1283.
@xhochy xhochy force-pushed the fix-trailing-space-in-string-statistics branch from 57cdb6f to f702f18 Compare May 1, 2018 13:28
@pitrou
Copy link
Member

pitrou commented May 1, 2018

I opened a competing PR #1973 to get CI as soon as possible. Let's see which one ends first.

@xhochy
Copy link
Member

xhochy commented May 1, 2018

@pitrou you should also have the right to push to contributor branches on github, no need to open new PRs if it's only a rebase.

@pitrou
Copy link
Member

pitrou commented May 1, 2018

In the past I've messed up things while trying to push to contributor branches :-)

@pitrou
Copy link
Member

pitrou commented May 1, 2018

Also the goal is to get an AppVeyor build on my account.

@pitrou
Copy link
Member

pitrou commented May 1, 2018

AppVeyor build succeeded at https://ci.appveyor.com/project/pitrou/arrow/build/1.0.359

@pitrou pitrou closed this in a3aaff5 May 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants