MINOR: Extends RocksDB docs#10046
Merged
Merged
Conversation
We recommend users to switch to jemalloc for RocksDB
mjsax
commented
Feb 3, 2021
| <span class="c1">// Cache and WriteBufferManager should not be closed here, as the same objects are shared by every store instance.</span> | ||
| <span class="o">}</span> | ||
| <span class="o">}</span> | ||
| </div> |
Member
Author
There was a problem hiding this comment.
Just some cleanup... (here and below). My editor flagged those...
JimGalasyn
reviewed
Feb 3, 2021
JimGalasyn
approved these changes
Feb 3, 2021
JimGalasyn
left a comment
Contributor
There was a problem hiding this comment.
LGTM, with one suggestion.
ableegoldman
reviewed
Feb 3, 2021
| <code class="docutils literal"><span class="pre">rocksdb.config.setter</span></code> configuration.</li> | ||
| <code class="docutils literal"><span class="pre">rocksdb.config.setter</span></code> configuration.</p> | ||
| <p>We further recommend, to change RocksDB's default memory allocator, because the default allocator may lead to increased memory consumption. | ||
| To change the memory allocator to <code>jemalloc</code>, you need to set the an environment variable before you start your Kafka Streams application:</p> |
Member
There was a problem hiding this comment.
Suggested change
| To change the memory allocator to <code>jemalloc</code>, you need to set the an environment variable before you start your Kafka Streams application:</p> | |
| To change the memory allocator to <code>jemalloc</code>, you need to set the environment variable before you start your Kafka Streams application:</p> |
Member
Author
There was a problem hiding this comment.
I guess it should be set an environment variable as we don't include the name?
Or should we change it to set the environment variable LD_PRELOAD?
Member
There was a problem hiding this comment.
Mm good point. I think set the environment variable LD_PRELOAD is best
Co-authored-by: Jim Galasyn <jim.galasyn@confluent.io>
rodesai
reviewed
Feb 4, 2021
| # example: install jemalloc (on Debian) | ||
| $ apt install -y libjemalloc-dev | ||
| # set LD_PRELOAD before you start your Kafka Streams application | ||
| $ LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so” |
Contributor
There was a problem hiding this comment.
just doing LD_PRELOAD= will only set the variable for that one command, which isn't doing anything. This should either be:
$ export LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so"
$ <application>
OR
$ LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libjemalloc.so" <application>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We recommend users to switch to jemalloc for RocksDB
Call for review @ableegoldman @JimGalasyn @rodesai