"Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode #7118

stonebig · 2019-08-29T12:44:45Z

I have a strange message, since around Jupyterlab-1.0.x:

Fatal process OOM in insufficient memory to create an Isolate

any idea/suggestion ?

[LabBuildApp] Building in C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\share\jupyter\lab
[LabBuildApp] Node v10.16.3

[LabBuildApp] Building jupyterlab assets (build:prod:minimize)
[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js install --non-interactive
[LabBuildApp] yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@1.2.9: The platform "win32" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning "jupyter-leaflet > leaflet.markercluster@1.4.1" has incorrect peer dependency "leaflet@~1.3.1".
warning "@jupyterlab/vega3-extension > vega-embed > vega-themes@2.3.2" has unmet peer dependency "vega@*".
warning "@jupyterlab/vega3-extension > vega-embed > vega-lite@2.7.0" has unmet peer dependency "vega@^3.0.0 || ^4.0.0".
[5/5] Building fresh packages...
success Saved lockfile.
Done in 192.13s.

[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js yarn-deduplicate -s fewer
[LabBuildApp] yarn run v1.15.2
$ C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\share\jupyter\lab\staging\node_modules\.bin\yarn-deduplicate -s fewer
Done in 2.52s.

[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js run build:prod:minimize
[LabBuildApp] yarn run v1.15.2
$ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.minimize.config.js

#
# Fatal process OOM in insufficient memory to create an Isolate
#

The text was updated successfully, but these errors were encountered:

stonebig · 2019-08-29T16:22:31Z

current list of complement shall be:

        @jupyter-widgets/jupyterlab-manager v1.0.2 enabled  ok
        @jupyterlab/vega3-extension v1.0.0 enabled  ok
        @pyviz/jupyterlab_pyviz v0.8.0 enabled  ok
        bqplot v0.4.7 enabled  ok
        jupyter-leaflet v0.11.1 enabled  ok
        jupyter-matplotlib v0.4.2 enabled  ok
        jupyter-threejs v2.1.1 enabled  ok
        jupyterlab-datawidgets v6.2.0 enabled  ok
        jupyterlab_bokeh v1.0.0 enabled  ok

What shall I do to recover this ?

reduce list of plugins ?
upgrade in yarn/npm version ?
is it the result of a bug ?

... maybe it's voila addition

stonebig · 2019-08-29T16:24:19Z

I read "It's a common issue when using TypeScript 2.1+ and webpack." https://www.npmjs.com/package/increase-memory-limit

stonebig · 2019-08-29T18:26:47Z

no more success, suggestions appreciated:


[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js run build:prod:minimize
[LabBuildApp] yarn run v1.15.2
$ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.minimize.config.js

#
# Fatal process OOM in insufficient memory to create an Isolate
#


<--- Last few GCs --->


<--- JS stacktrace --->

error Command failed with exit code 3221225477.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\debuglog.py", line 47, in debug_logging
    yield

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\labapp.py", line 96, in start
    core_config=self.core_config)

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\commands.py", line 378, in build
    command=command, clean_staging=clean_staging)

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\commands.py", line 583, in build
    raise RuntimeError(msg)

[LabBuildApp] RuntimeError: JupyterLab failed to build

[LabBuildApp] Exiting application: JupyterLab

stonebig · 2019-08-29T18:30:34Z

is this "--max_old_space_size=4096" a problem on 32 bit ? seeing microsoft/vscode#43223 (comment)

stonebig · 2019-08-29T18:36:19Z

so trying back to Jupyterlab-1.0.1 , before that 4096 , to check if diagnostic is rather
"hard to rebuild Jupyterlab-1.02 and next, on Win10 / 32 bit mode"

jasongrout · 2019-08-29T18:45:01Z

Also, does it help to use jlab 1.1.0 with the new jupyter lab build --minimize=False?

jasongrout · 2019-08-29T18:46:37Z

Also, you may be right that setting the memory to 4GB is too much for 32-bit? You can try setting it lower and see if it helps.

stonebig · 2019-08-29T18:53:13Z

hi @jasongrout , I will try now your suggestion "--minimize=False" + Jupyterlab-1.1.0

and how to play with the memory parameter ?

jasongrout · 2019-08-29T19:13:50Z

Use jupyter lab path to find your application directory, and inside that directory, inside the staging directory, you'll find the package.json jlab uses to build, and there you should find these lines:

jupyterlab/jupyterlab/staging/package.json

Lines 9 to 11 in e7d5441

    
           "build:prod": "cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.config.js", 
        
           "build:prod:minimize": "cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.minimize.config.js", 
        
           "build:prod:stats": "cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --profile --config webpack.prod.minimize.config.js --json > stats.json",

You can try adjusting it there

stonebig · 2019-08-29T20:38:51Z

--minimize=false was not a success:

[LabBuildApp] Building in C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\share\jupyter\lab
[LabBuildApp] Node v10.16.3

[LabBuildApp] Building jupyterlab assets (build:prod)
[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js install --non-interactive
[LabBuildApp] yarn install v1.15.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
info fsevents@1.2.9: The platform "win32" is incompatible with this module.
info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning "jupyter-leaflet > leaflet.markercluster@1.4.1" has incorrect peer dependency "leaflet@~1.3.1".
warning "@jupyterlab/vega3-extension > vega-embed > vega-themes@2.4.0" has unmet peer dependency "vega@*".
warning "@jupyterlab/vega3-extension > vega-embed > vega-lite@2.7.0" has unmet peer dependency "vega@^3.0.0 || ^4.0.0".
[5/5] Building fresh packages...
success Saved lockfile.
Done in 195.11s.

[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js yarn-deduplicate -s fewer
[LabBuildApp] yarn run v1.15.2
$ C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\share\jupyter\lab\staging\node_modules\.bin\yarn-deduplicate -s fewer
Done in 2.54s.

[LabBuildApp] > node C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\staging\yarn.js run build:prod
[LabBuildApp] yarn run v1.15.2
$ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.config.js

#
# Fatal process OOM in insufficient memory to create an Isolate
#


<--- Last few GCs --->


<--- JS stacktrace --->

error Command failed with exit code 3221225477.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[LabBuildApp] JupyterLab failed to build
[LabBuildApp] Traceback (most recent call last):

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\debuglog.py", line 47, in debug_logging
    yield

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\labapp.py", line 96, in start
    core_config=self.core_config)

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\commands.py", line 378, in build
    command=command, clean_staging=clean_staging)

[LabBuildApp]   File "C:\WinP\bd37\bu\WPy32-3741b2\python-3.7.4\lib\site-packages\jupyterlab\commands.py", line 583, in build
    raise RuntimeError(msg)

[LabBuildApp] RuntimeError: JupyterLab failed to build

[LabBuildApp] Exiting application: JupyterLab

stonebig · 2019-08-29T20:46:10Z

trying "jupyter lab build" with "NODE_OPTIONS=--max_old_space_size=3000"

it worked....

how to make it more auto-adjustable ? rather than having to go hack into package.json ?

stonebig · 2019-08-29T21:19:50Z

now retrying with voila and max_old_space_size=3000 for 32 bit

vidartf · 2019-08-29T23:41:00Z

Potentially also --dev-build=True can help.

dhirschfeld · 2019-08-29T23:53:13Z

I have a vague recollection that this may just be an out-of -memory error on your PC i.e. independent from node.

If you're running 32bit on Windows you may simply be out of luck.

One thing to try is that by default 32bit Windows can only address 2GB of memory. You can however increase that to 3GB with the following hack:
https://knowledge.autodesk.com/support/autocad/learn-explore/caas/sfdcarticles/sfdcarticles/How-to-enable-a-3GB-switch-on-Windows-Vista-Windows-7-or-Windows-XP-s.html

https://techcommunity.microsoft.com/t5/Ask-The-Performance-Team/Memory-Management-Demystifying-3GB/ba-p/372333

I think the application may have to opt-in to this so YMMV.

stonebig · 2019-08-30T05:02:52Z

so, with voila it fails again.:

I'll remove voila from my 32bit build,
I'll suppose that soon Jupyter will become too big for 32 bit Windows.

After Python-2, maybe an E.O.L for 32 bit Windows shall be decided.

jasongrout · 2019-08-30T17:53:10Z

So it sounds like this investigation is resolved now?

stonebig · 2019-08-30T18:49:09Z

root cause is known, a slow disease with no long term remedies, except maybe a modular Jupyterlab-2

jasongrout · 2019-08-30T18:56:43Z

Thanks! Closing as answered, then. Sorry you have to still support 32-bit windows.

stonebig · 2019-08-31T11:21:55Z

I will just use plain Jupyterlab in 32bit mode, no attempt to rebuild it with extensions.

vidartf · 2019-08-31T13:12:25Z

You could pack your own distro on a 64-bit machine and install that on the 32 bit systems, if needed.

stonebig · 2019-08-31T13:29:18Z

I use a 64 bit PC and a 32 bit tool chain.

If it doesn't build "naturally" that way, it won't "re-build" on the end-user 32bit PC, making the solution not self-maintainable.

stonebig changed the title ~~"iFatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode~~ "Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode Aug 29, 2019

stonebig mentioned this issue Aug 29, 2019

release 2019-03 follow-up winpython/winpython#764

Closed

7 tasks

jasongrout added this to the Reference milestone Aug 30, 2019

jasongrout closed this as completed Aug 30, 2019

lock bot added the status:resolved-locked label Oct 7, 2019

lock bot locked as resolved and limited conversation to collaborators Oct 7, 2019

"Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode #7118

"Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode #7118

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 •

edited

jasongrout commented Aug 29, 2019

jasongrout commented Aug 29, 2019

stonebig commented Aug 29, 2019 •

edited

jasongrout commented Aug 29, 2019

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

vidartf commented Aug 29, 2019

dhirschfeld commented Aug 29, 2019

stonebig commented Aug 30, 2019

jasongrout commented Aug 30, 2019

stonebig commented Aug 30, 2019 •

edited

jasongrout commented Aug 30, 2019

stonebig commented Aug 31, 2019

vidartf commented Aug 31, 2019

stonebig commented Aug 31, 2019

"Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode #7118

"Fatal process OOM in insufficient memory to create an Isolate" trying to build Jupyterlab on Win10 / 32 bit mode #7118

Comments

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 • edited

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 • edited

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 • edited

jasongrout commented Aug 29, 2019

jasongrout commented Aug 29, 2019

stonebig commented Aug 29, 2019 • edited

jasongrout commented Aug 29, 2019

stonebig commented Aug 29, 2019

stonebig commented Aug 29, 2019 • edited

stonebig commented Aug 29, 2019 • edited

vidartf commented Aug 29, 2019

dhirschfeld commented Aug 29, 2019

stonebig commented Aug 30, 2019

jasongrout commented Aug 30, 2019

stonebig commented Aug 30, 2019 • edited

jasongrout commented Aug 30, 2019

stonebig commented Aug 31, 2019

vidartf commented Aug 31, 2019

stonebig commented Aug 31, 2019

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 29, 2019 •

edited

stonebig commented Aug 30, 2019 •

edited