Why not p-limit

I did not dig through it, but in my case, it seems p-limit will cause the whole process turn into serial, which is actually much slower.
This is the case manually grouped by 50:

This is the case using p-limit, single task time cost is printed, the max concurrency is set to 50, then t/50 maybe the actual time cost.

Hello, @troyeagle first congrats by this job, its really cool.

I'm trying to optimize this flow or trying understand better.
I have a blog with >2K posts and the memory use on build is so big.
I'm trying setting up some different max concurrency but without success yet.

I don't know if do you remember but, on my tests, the memory is never cleanuped while the loop run.
I'm trying cleanup the memory while the process is running but I can´t do it.
In other words, I can to limit the async process with the max concurrency, but the memory usage is more big each time.
This is a dump of memory after each promise call.

using max-concurrency 5 for try: the memory growth slow but never go back

My tests there are around this place:

I tried set as null the variables to 'clean-up' memory, but I guess the GC do it alone. BUT if do, is not doing.

Thanks for your attention, if you have some idea to share with me, will be great.

As I remember, what magic happens may be in the method this.renderPage, or your processAsyncRenderPage which uses vue-server-renderer and use its renderString utility.

The server-renderer may be stateful, it not just accepts an object and uses template to parse it, but run with a context and then create a bunch of async rendering tasks. The context may expand during the rendering process to save building cache for optimizing following tasks' rendering performance.

I think it's not the results' not being GC-ed but the context growing that causes your memory growing. It seems vuepress always uses one context, when a group of tasks ends, just some of the spaces are set free. @sr2ds You could check whether the total size of 5 pages' html output string is the same as your memory growth per time, or use chrome://inspect to see what object in the memory is really growing as the task goes on, then you may have idea of what objects or attributes you should set null to trigger garbage collection.

In all, the memory problem is the tradeoff of server-side rendering optimization. Vuepress maybe did not expect such huge size of pages, and in vuepress build case, building speed may be the first concern.

Thanks so much for your time and answer.

To be honest I don't know how do a good review on inspect mode, but looks that the string type of data is the most used, I don't sure if can be the returns of renderString stored on context.

I made some tests but without success yet.

I saw this other issue with some comparatives between vuepress and vitepress, I'll try a little bit more but looks more prudent to migrate to vitepres on this specific blog case.