Skip to content

babel-register cache grows infinitely and breaks v8 #5667

@jamietre

Description

@jamietre

Choose one: is this a bug report or feature request? a bug

Expected Behavior

By default, babel-register creates a cache in the user's home directory, .babel.json. This cache appears to be unmanaged based on looking at ./babel-register/lib/cache.js. The cache should manage itself to avoid growing to an extremely large size.

Current Behavior

I started experiencing v8 crashes when running mocha tests using --compilers js:babel-core/register as below:

<--- Last few GCs --->

   82518 ms: Mark-sweep 807.1 (1039.7) -> 802.3 (1038.7) MB, 149.2 / 0.0 ms [allocation failure] [GC in old space requested].
   82668 ms: Mark-sweep 802.3 (1038.7) -> 802.3 (1036.7) MB, 150.6 / 0.0 ms [allocation failure] [GC in old space requested].
   82838 ms: Mark-sweep 802.3 (1036.7) -> 802.2 (993.7) MB, 169.7 / 0.0 ms [last resort gc].
   82989 ms: Mark-sweep 802.2 (993.7) -> 802.2 (982.7) MB, 150.6 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000024EE58CFB61 <JS Object>
    1: SparseJoinWithSeparatorJS(aka SparseJoinWithSeparatorJS) [native array.js:~75] [pc=000002B8298FC057] (this=0000024EE5804381 <undefined>,w=0000011715C4D061 <JS Array[7440]>,F=000003681BBC8B19 <JS Array[7440]>,x=7440,I=0000024EE58B46F1 <JS Function ConvertToString (SharedFunctionInfo 0000024EE5852DC9)>,J=000003681BBC8AD9 <String[4]\: ,\n  >)
    2: DoJoin(aka DoJoin) [native array.js:137...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

I traced these to babel-register/lib/cache.js code calling JSON.stringify on the cache object.

  try {
    serialised = (0, _stringify2.default)(data, null, "  ");
  } catch (err) {

My .cache.json was over 200 megabytes. Deleting it immediately resolved the problem.

Possible Solution

  • cache should periodically expire old things and have a maximum size
  • cache could be implemented using some kind of simple database that's more efficient than reading the entire cache into memory & rewriting it at the end of a session

Context

Prevents inline transpilation from working properly, and performance suffers significantly as the cache size grows and each operation requires reading/writing a huge file.

Because it's very difficult to trace the source of v8 crashes, this is a rather insidious bug. There is at least one other bug report in a random package that is almost certainly this issue:

caolan/async#1311

This would primarily become an issue for people running large test suites using babel-register in a single environment that is never purged (e.g. a dev workstation). I expect even though it may not manifest very often, there are certainly performance and stability implications for a large number of users for never pruning the cache.

Your Environment

Windows 10
Node 6.10.2
Npm 4.2.0
Babel 6.18.2

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions