Skip to content

Memory leak in LRU cache? #27

@ankgup87

Description

@ankgup87

Hi,

I am seeing a strange memory leak issue with LRUCache. When LRUCache capacity is set as 8MB, then process's memory space remain constant and does not grow with time. When LRUCache's capacity is set > 8MB then process's memory space slowly grows and is ultimately killed by OS.

DB used for testing

DB already has 650M rows where each row's key is around 20 bytes and value averaging around 250 bytes. DB operations are put and multiGet. Keys used for multi-gets are chosen in random fashion. Rocksdb is loaded as part of JAVA process and DB operations are accessed through JNI layer. Cache is not exposed through JNI layer and default cache used by rocksdb is used.

Cache testing

  1. Default cache of rocksdb (8MB capacity) is used and process's memory size remains constant.
  2. When cache size is bumped up to say 512MB, process's memory space slowly grows and is finally killed by OS. Cache capacity was bumped up by changing db/db_impl.cc:
    result.block_cache = NewLRUCache(67108864);

Please note that between 1) and 2), there were no changes in code other than increasing cache capacity.

Can someone please comment if they have seen this behavior? Thanks.!

PS: I am using latest code of rocksdb from github repo. Other options used for creating DB:

Write buffer size: 128MB
DisableSeekCompaction: false
block size: 4KB
maxWriteBufferNumber: 3
Background compaction threads: 5
level0 file num compaction trigger: 4
level 0 stop files trigger: 12
numLevels: 4
maxGrandparent Overlap Factor: 10
Max bytes for level base: 1GB
No Compression
Custom compaction filter

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions