Skip to content

Make npm api requests in parallel #181

@Mic92

Description

@Mic92

It seems that node2nix currently performs blocking requests:

diff --git a/lib/sources/NPMRegistrySource.js b/lib/sources/NPMRegistrySource.js
index 57d99be..81c627f 100644
--- a/lib/sources/NPMRegistrySource.js
+++ b/lib/sources/NPMRegistrySource.js
@@ -79,7 +79,9 @@ NPMRegistrySource.prototype.fetch = function(callback) {
               };
             }
 
+            console.log(`start downloading ${url}`)
             client.get(url, clientParams, function(err, data, raw, res) {
+                console.log(`finished downloading ${url}`)
                 if(err) {
                     callback(err);
                 } else if(data == undefined || data.versions === undefined) {
$ ./result/bin/node2nix
fetching local directory: ./. from .
start downloading https://registry.npmjs.org/optparse
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/optparse
http 200 https://registry.npmjs.org/optparse
finished downloading https://registry.npmjs.org/optparse
start downloading https://registry.npmjs.org/semver
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/semver
http 200 https://registry.npmjs.org/semver
finished downloading https://registry.npmjs.org/semver
start downloading https://registry.npmjs.org/npm-registry-client
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/npm-registry-client
http 200 https://registry.npmjs.org/npm-registry-client
finished downloading https://registry.npmjs.org/npm-registry-client
start downloading https://registry.npmjs.org/npmconf
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/npmconf
http 200 https://registry.npmjs.org/npmconf
finished downloading https://registry.npmjs.org/npmconf
start downloading https://registry.npmjs.org/tar
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/tar
http 200 https://registry.npmjs.org/tar
finished downloading https://registry.npmjs.org/tar
start downloading https://registry.npmjs.org/temp
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/temp
http 200 https://registry.npmjs.org/temp
finished downloading https://registry.npmjs.org/temp
start downloading https://registry.npmjs.org/fs.extra
info attempt registry request try #1 at 14.34.44
http request GET https://registry.npmjs.org/fs.extra
http 200 https://registry.npmjs.org/fs.extra
finished downloading https://registry.npmjs.org/fs.extra
start downloading https://registry.npmjs.org/findit

Maybe you can explain why this happens as you also seem to maintain the asynchronous abstraction behind it: https://github.com/svanderburg/slasp

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions