Skip to content
This repository was archived by the owner on Dec 1, 2024. It is now read-only.
/ rocksdb Public archive
This repository was archived by the owner on Dec 1, 2024. It is now read-only.

Memory leak in db.getMany() #192

@tzapzoor

Description

@tzapzoor

Context:
OS: macOS 12.2.1
Chip: Apple M1 Pro
Node: v16.13.2
rocksdb: 5.2.0
level-rocksdb: 5.0.0

Db options:

const level = require('level-rocksdb');

const db = level('./database', {
    valueEncoding: 'json',
    createIfMissing: false,
    readOnly: true,
});

I stress tested the db.GetMany method with batches of 500 keys as part of a simple node.js express app and after some time the memory usage of process was about 40GB and growing.

let keys: Buffer[] = getKeys(req.keys);
let values: (Value | undefined)[] = await db.getMany(keys);

I then replaced getMany with get and the memory stabilised at about 300MB, with worse performance, of course.

let values: (Value | undefined)[] = await Promise.all(
keys.map(async (k): Promise<(Value | undefined)> => {
    try {
        return await db.get(k);
    } catch(e) {
        return undefined;
    }
}));

The problem reproduces 100%. Do you guys have ideas why this could happen?

cc: @vweevers pinging as I saw you added support for getMany.

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions