ARROW-739: Don't install jemalloc in parallel #456
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
wesm
reviewed
Mar 30, 2017
cpp/CMakeLists.txt
Outdated
| BUILD_IN_SOURCE 1 | ||
| BUILD_COMMAND ${MAKE}) | ||
| BUILD_COMMAND ${MAKE} | ||
| INSTALL_COMMAND ${MAKE} -j1) |
Member
There was a problem hiding this comment.
I guess this works without install?
Member
Author
There was a problem hiding this comment.
No, it should include install. I did not realise that we have libjemalloc already installed in Travis.
Contributor
|
On my machine (Mac OS) and on Linux, the new code failed with I guess Now I'm investigating if the combination fixes the original problem. |
Contributor
|
After this change 820035e, the problems seems to have gone away. I've compiled arrow about 100 times on that branch and didn't see the error. |
Member
|
Cool, that's great. Fun times with CMake as always. We'll pull in that change and merge the patch tomorrow |
wesm
approved these changes
Mar 31, 2017
Member
wesm
left a comment
There was a problem hiding this comment.
+1, all good locally on CMake 2.8.x. Will await green build
Member
|
The build failed because this patch is behind the Parquet API changes -- libarrow_jemalloc built fine though. Merging, thanks! |
wesm
pushed a commit
to wesm/arrow
that referenced
this pull request
Sep 8, 2018
…second timestamp with arrow writer properties set to coerce timestamps and support deprecated int96 timestamps. The bug was a due to the fact that the physical type was int64 but the WriteTimestamps function was taking a path that assumed the physical type was int96. This caused memory corruption because it was writing past the end of the array. The bug was fixed by checking that coerce timestamps is disabled when writing int96. A unit test was added for the regression. Author: Joshua Storck <joshua.storck@twosigma.com> Closes apache#456 from joshuastorck/ARROW_2082 and squashes the following commits: 5fa0a94 [Joshua Storck] Removing 'using ::arrow' in favor of using ::arrow::SomeType 9725ecc [Joshua Storck] Bug fix for ARROW-2082, in which a segfault was being encountered when writing a nanosecond timestamp column with arrow writer properties set to coerce timestamps and support deprecated int96 timestamps. The bug was a segfault due to the fact that the physical type was int64 but the WriteTimestamps function was taking a path that assumed the physical type was int96. The bug was fixed by checking that coerce timestamps is disabled when writing int96. A unit test was added for the regression Change-Id: I9551796c415416fd1e64890b32fe4a59db2fb825
This was referenced Aug 18, 2020
kou
pushed a commit
that referenced
this pull request
Aug 19, 2020
On ARROW-6437 (#7928) we saw occasional "File exists" errors on `jemalloc_ep` on macOS. Googling the error message led back to ARROW-739 (#456), which fixed this before by forcing install with `-j1`. ARROW-3492 later made it so jemalloc would build (but not install) in parallel. Then ARROW-3539 (#2779) was addressing a problem with that change and, along with fixing the build parallelization issue, unfortunately reverted the original `make -j1 install` fix. This patch restores the fix from ARROW-739. Closes #7995 from nealrichardson/jemalloc-install Authored-by: Neal Richardson <neal.p.richardson@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
pribor
pushed a commit
to GlobalWebIndex/arrow
that referenced
this pull request
Oct 24, 2025
## What's Changed Derived from the ADBC one. Fixes apache#456. --------- Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Alternative fix proposal. Couldn't trigger it locally though.