Skip to content

Default JSON loader uses too much memory #15789

@Nate-Wessel

Description

@Nate-Wessel

Bug report

What is the current behavior?
I'm working on a project that depends on a lot of topojson files (~25MB currently). These are all valid JSON files, but we had used the file-extension *.topojson for them. Initially we had these included as an asset/resource and were getting them with fetch. I realized it made more sense to use a dynamic import and get their data in chunks; I renamed all the files to *.topo.json so that WebPack would recognize them as JSON. This quickly produced out of memory errors in development mode similar to those described in #1914 as my memory usage exceeded 3.3GB vs. the ~250MB typically needed before the change.

I somehow realized that explicitly defining json-loader as the loader for these files would eliminate the problem. With the same files imported in this way, memory usage after the initial development build is down around 370MB; bigger than before of course, but well under the 3GB when using the built in JSON file loader.

If the current behavior is a bug, please provide the steps to reproduce.
My working code is here prior to the fix vs here one changeset later with the loader explicitly specified and the files renamed.

What is the expected behavior?
Required JSON files should require memory in proportion to their size. ~3GB extra RAM needed for 25MB of JSON is not right.

Other relevant information:
webpack version: 5.21.2 (tried with latest as well, no difference)
Node.js version: 14.19.2
Operating System: Ubuntu 20
Additional tools:

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions