Skip to content

queue.jobs memory leak #54

@jbielick

Description

@jbielick

I believe this line leaks memory.

I'm using this in an infinitely long-running application that continuously pushes jobs to redis to be processed by other threads. The app tails a MySQL replication log, so it's never-ending.

I noticed a memory leak because my node process was consuming as much memory as redis itself (where all the queued messages are stored). Redis and the node process's memory usage were essentially 1:1.

I noticed this line which stores the jobs on the queue object indexed by the job id. Over time, would this not leak memory on the queue object? These are never garbage collected. Over a period of 20 minutes or so, this can increase to 400mb.

The memory leak appeared to come from this library for two reasons:
1.) when I commented out the queue.createJob, the processes only used ~30mb of memory continuously.
2.) Using memwatch and heapdump, I exported a heap snapshot after a memory leak was detected (mem usages increased consistently after 5 garbage collection events) and upon viewing the results, queue.jobs accounted for 92% of the heap usage

screen shot 2016-11-30 at 9 22 17 pm

I'm not super familiar with this lib. Is this line crucial or can it be removed? I'll probably just clear this array manually as I'm not using progress, events or callbacks. Just need to fire and forget.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions