Skip to content

refactor(ls): greatly simplify ls implimentation#369

Merged
ariporad merged 2 commits intomasterfrom
simplify-ls
Mar 6, 2016
Merged

refactor(ls): greatly simplify ls implimentation#369
ariporad merged 2 commits intomasterfrom
simplify-ls

Conversation

@ariporad
Copy link
Copy Markdown
Contributor

This greatly simplifies the implementation of ls, and probably makes it a good deal more performant.

@nfischer
Copy link
Copy Markdown
Member

@ariporad does this work for ls('-d')? I noticed you cut out the mention of options.directory, which makes me suspicious this could work.

Also, this breaks some additional tests, so those should be resolved first.

@ariporad
Copy link
Copy Markdown
Contributor Author

@nfischer: So, we appear to have some inconsistent ls behavior. If your pass in a file/pattern with no matches, should it have an error, or just return an array with nothing in it?

@nfischer
Copy link
Copy Markdown
Member

@ariporad I accounted for this in common.expand(). In bash:

$ ls this*file*doesnt*exist.txt
ls: cannot access this*file*doesnt*exist.txt: No such file or directory
# bash passes a literal 'this*file*doesnt*exist.txt' to the ls executable

@ariporad
Copy link
Copy Markdown
Contributor Author

@nfischer: So if it can't find the file it should error?

@nfischer
Copy link
Copy Markdown
Member

That's what ls() should do, yes. That's why I use common.expand() to inside common.wrap(). It will pass non-existing file names to commands (like ls()). Then those commands are expected to handle the non-existing file themselves (in ls()'s case, it should output an error message).

@nfischer
Copy link
Copy Markdown
Member

glob.sync() returns an empty array, I think, which probably isn't what you want

@nfischer
Copy link
Copy Markdown
Member

@ariporad pinging this to see if there's any work being done to re-add support for -l and -d.

@ariporad
Copy link
Copy Markdown
Contributor Author

@nfischer: Sorry, I've been sick. I'll work on that ASAP.

@ariporad
Copy link
Copy Markdown
Contributor Author

@nfischer: Both -l and -d should be back now.

shell.js Outdated
//@include ./src/ls
var _ls = require('./src/ls');
exports.ls = common.wrap('ls', _ls, {idx: 1});
exports.ls = common.wrap('ls', _ls, {noGlob: true});
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this taken out? Also, does noGlob even do anything here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to glob in common.wrap, we want to do it in late itself, because we want to optionally pass different opts to common.expand depending on the flags passed in.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think noGlob is a real option (no idx value will have the same effect)

Also, would it be possible to modify common.expand() to take all as an argument? It might be cleaner to keep glob inside common.js

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a real option.

Moving the all option into wrap would be really complex, and probobly not that useful. ls needs to do some weird stuff with common.expand, so I think we should just keep it in ls.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config.noGlob is real, options.noGlob isn't. You're specifying options.noGlob here (just like how idx is really options.index here)

@nfischer
Copy link
Copy Markdown
Member

@ariporad I'm going to make some adjustments to this branch and see if I can fix some of the errors we're seeing.

@ariporad
Copy link
Copy Markdown
Contributor Author

Ok, thanks! If not, I'll try to look into it tomorrow or when I get back.

@nfischer nfischer force-pushed the simplify-ls branch 6 times, most recently from 216f996 to 5df3ee2 Compare February 29, 2016 11:08
@nfischer
Copy link
Copy Markdown
Member

@ariporad Take a look now. I fixed the bug you had for Windows (you forgot /g on a regex, that's all). Nice PR!

I tested performance and saw that this was very very slow... So I did my best to squeeze out some more performance. This is about 2x faster than your initial revision (and about 2x uglier...). This is still slower (also about 2x...) than the master branch, but it should be a lot cleaner.

So it's a tradeoff. More maintainable and a bit slower, or we can stick with the tried and true method and get faster.

As for the tests, only one of them needed to change (the one you changed for ls('...*.j')). I just changed some others because I realized I could format them a bit nicer, make them more robust (the tests using idx were actually written poorly, my fault), and let them catch more.

I also fixed a but where ls -l resources/ls would return values with names including resources/ls (they should be relative to resources/ls, not the current directory), so that's what's going on there.

@nfischer
Copy link
Copy Markdown
Member

nfischer commented Mar 4, 2016

@ariporad This should be rebased off master. If you can, see if you find ways to improve performance (or refactor things into a function), since we're still taking a performance hit for the normal use case, which isn't ideal.

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 4, 2016

@nfischer: I'm going to try something, give me a second.

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 5, 2016

@nfischer: Actually, let me talk this through with you first.

Personally I vote for slower and more maintainable, it's not like ls is a big bottleneck.

It also might be better to try to hijack fs.stat, so common.wrap looks something like this:

function wrap(cmd, fn, options) {
  return function() {
    var oldStat = fs.stat;
    var statResults = {};
    fs.stat = function (file) {
        if (!statResults[file]) statResults[file] = oldStat.apply(this, arguments);
        return statResults[file];
    }

    // Whatever common.wrap used to do

    fs.stat = oldStat;
  };
}

I bet this would speed things up a bit.

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 5, 2016

Nope, no it wouldn't.

But I did learn how to benchmark node things.

@nfischer
Copy link
Copy Markdown
Member

nfischer commented Mar 5, 2016

@ariporad I just pushed another unit test to ls() to make sure everything is working nicely. Assuming everything still passes (it should), does this look good to merge?

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 5, 2016

LGTM!

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 5, 2016

Oh, wait, I have to rebase this off master.

@ariporad
Copy link
Copy Markdown
Contributor Author

ariporad commented Mar 6, 2016

LGTM

ariporad added a commit that referenced this pull request Mar 6, 2016
refactor(ls): greatly simplify ls implimentation
@ariporad ariporad merged commit cbbdb79 into master Mar 6, 2016
@ariporad ariporad deleted the simplify-ls branch March 6, 2016 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants