Remove mmapSize check, it has wrong value on ARM64#281
Merged
manishrjain merged 1 commit intodgraph-io:masterfrom Sep 1, 2021
Merged
Remove mmapSize check, it has wrong value on ARM64#281manishrjain merged 1 commit intodgraph-io:masterfrom
manishrjain merged 1 commit intodgraph-io:masterfrom
Conversation
|
|
manishrjain
approved these changes
Sep 1, 2021
Contributor
manishrjain
left a comment
There was a problem hiding this comment.
Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status:complete! all files reviewed, all discussions resolved (waiting on @yarysh)
46bit
added a commit
to 46bit/distributed-systems
that referenced
this pull request
Sep 18, 2021
I hope this will fix the following critical error that I have seen whenever writing a lot of data to Badger on my Raspberry Pi: Unable to truncate file: "[…]/000001.vlog" error: mremap size mismatch: requested: 271382146 got: 536870912 This error causes data loss (yikes.) It seems to have been fixed in Ristretto already [1] so I'm going to try that. [1] dgraph-io/ristretto#281
46bit
added a commit
to 46bit/distributed-systems
that referenced
this pull request
Sep 18, 2021
I hope this will fix the following critical error that I have seen whenever writing a lot of data to Badger on my Raspberry Pi: Unable to truncate file: "[…]/000001.vlog" error: mremap size mismatch: requested: 271382146 got: 536870912 This error causes data loss (yikes.) It seems to have been fixed in Ristretto already [1] so I'm going to try that. [1] dgraph-io/ristretto#281
This was referenced Dec 9, 2021
|
@manishrjain could you please release this fix? It has been merged almost half a year ago, and there is still no release with a fix. |
Contributor
|
Hi. I’m no longer maintaining these repos, as I’m no longer with Dgraph Labs. We have fork of this repo that we maintain at outcaste-io. Feel free to file an issue there and we can make a release. |
This was referenced Jun 6, 2022
lithp
pushed a commit
to lithp/ristretto
that referenced
this pull request
Jun 21, 2022
bothra90
added a commit
to fennel-ai/rexer
that referenced
this pull request
Aug 18, 2022
This commit updates ristretto to HEAD@master to pull in the fix for arm: dgraph-io/ristretto#281
bothra90
added a commit
to fennel-ai/rexer
that referenced
this pull request
Aug 19, 2022
This commit updates ristretto to HEAD@master to pull in the fix for arm: dgraph-io/ristretto#281
bothra90
added a commit
to fennel-ai/rexer
that referenced
this pull request
Aug 19, 2022
This commit updates ristretto to HEAD@master to pull in the fix for arm: dgraph-io/ristretto#281
bothra90
added a commit
to fennel-ai/rexer
that referenced
this pull request
Aug 19, 2022
This commit updates ristretto to HEAD@master to pull in the fix for arm: dgraph-io/ristretto#281
bothra90
added a commit
to fennel-ai/rexer
that referenced
this pull request
Aug 19, 2022
This commit updates ristretto to HEAD@master to pull in the fix for arm: dgraph-io/ristretto#281
joshua-goldstein
pushed a commit
that referenced
this pull request
Sep 18, 2022
(cherry picked from commit c2901dc)
joshua-goldstein
pushed a commit
that referenced
this pull request
Oct 10, 2022
This PR adds a specific arm64 implementation for the `mremap` function which ignores the second return argument from the syscall. **All other Linux implementations are unchanged.** This PR reworks [an earlier PR](#281) so that the functionality on non-arm64 platforms remains the same. Tested on Linux amd64 and Linux arm64.
mangalaman93
pushed a commit
that referenced
this pull request
Dec 11, 2023
In #281 mmap size check has been disabled for the ARM64 arch. But we at [anytype](https://github.com/anyproto) quite often face the same problem on the arm arch on Android devices (armeabi-v7a target) with the [latest badger v4.2.0](https://github.com/dgraph-io/badger/releases/tag/v4.2.0) and [latest ristretto v0.1.1](https://github.com/dgraph-io/ristretto). ```memtables error: while opening fid: 1 error: while updating skiplist error: mremap size mismatch: requested: 1479692 got: 67108864``` This PR extends the #281 to cover all arm architectures. Here is one more mention of this issue dgraph-io/badger#2027 (comment) It would also make sense to create the PR on top of v0.1.1 release, but there is not release branch to base on
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this statement:
mmapAddr, mmapSize, errno := unix.Syscall6(unix.SYS_MREMAP,...)mmapSize- it is not an allocated size, at least for ARM64 arch.For ARM64,
mmapSizewill be equal to the old size.Example script: https://gist.github.com/yarysh/2b4db19ebe1236f31ff6780d956628f5
I run this script on AMD64 machine with Ubuntu 20.04, and it returns:
And on ARM64 machine (Raspberry PI and AWS Graviton) with Ubuntu 20.04, it returns:
I have checked man for mremap and it describes returns as:
This issue also affected badger on ARM64: https://discuss.dgraph.io/t/error-mremap-size-mismatch-on-arm64/15333
This change is