Skip to content

[WIP] allow global_defs to pass in objects and arrays#1467

Closed
alexlamsl wants to merge 1 commit intomishoo:masterfrom
alexlamsl:global_defs-unsafe
Closed

[WIP] allow global_defs to pass in objects and arrays#1467
alexlamsl wants to merge 1 commit intomishoo:masterfrom
alexlamsl:global_defs-unsafe

Conversation

@alexlamsl
Copy link
Copy Markdown
Collaborator

This is an alternative approach to #1198 which is based upon #1425 to fix #1416. We extend global_defs to allow object as input values, which it would previously throw an error upon compression.

With this PR global_defs now requires evaluate (true by default) for substitutions to take place. Also the evaluate() caching that was implemented in #1425 is now used in general.

@@ -1,29 +0,0 @@
do_not_update_lhs: {
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only file with "extensive" testing for global_defs before (there is one in test/compress/reduce_vars.js as well).

I was hoping git will pick this up as a file rename, but this is basically copied verbatim to test/compress/global_defs.js with added evaluate:true.

this.walk(new TreeWalker(function(node){
if (node instanceof AST_SymbolRef) {
var d = node.definition();
if (d && d.init) {
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many places in the codebase where AST_Symbol.definition() is assumed to return defined, non-null value.

this._evaluating = true;
try {
var d = this.definition();
if (d && (d.constant || compressor.option("reduce_vars") && !d.modified) && d.init) {
Copy link
Copy Markdown
Collaborator Author

@alexlamsl alexlamsl Feb 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many places in the codebase where AST_Symbol.definition() is assumed to return defined, non-null value.

// }
// })).transform(compressor);

if (val instanceof AST_Node) return val.transform(compressor);
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining two usages of make_node_from_constant() will never pass in AST_Node as val.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Could you please add the positive and negative test case scenarios I mentioned in #1198?

#1198 (comment)
#1198 (comment)
#1198 (comment)

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

What happens in a mixed LHS and RHS scenario?

        DEBUG = 1;
        DEBUG++;
        DEBUG += 1;
        f(DEBUG);
        x = DEBUG;

Will all DEBUG replacements be disallowed with an appropriate warning?

Or at the very least a fatal error thrown upon first attempt to change a LHS expression.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

This PR breaks existing CLI usage:

uglify-js@2.7.5:

$ echo 'console.log(D);' | uglifyjs --define D=5 -cm
console.log(5);

this PR:

$ echo 'console.log(D);' | bin/uglifyjs --define D=5 -cm
console.log(D);

Could you please add a mocha test for the CLI --define option after this is fixed?

Also, how to set compress global_defines objects and arrays on the CLI?

Would prefer if existing test files are not renamed. That way it is easier to discern if the tests' behavior has been changed.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

What happens in a mixed LHS and RHS scenario?

Unfortunately that would replace the RHS cases (and leave the LHS ones alone) - this is the same behaviour as before this PR though.

Investigating the command-line breakage at the moment, will add mocha and those other tests as you mentioned above. Then I'll ponder about the mixed case and see how to go about fixing it. (or may be open another PR?)

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Unfortunately that would replace the RHS cases (and leave the LHS ones alone) - this is the same behaviour as before this PR though.

The current behavior is not great in this regard. Would prefer it to throw a fatal exception at the first attempt to set a LHS variable so the user is aware that their code must be fixed.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

The current behavior is not great in this regard. Would prefer it to throw a fatal exception at the first attempt to set a LHS variable so the user is aware that their code must be fixed.

Understood - you want this to be in the same PR or separate?

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

You can make the fixes in this PR I think.

Can you please add a [WIP] prefix?

@alexlamsl alexlamsl changed the title allow global_defs to pass in objects and arrays [WIP] allow global_defs to pass in objects and arrays Feb 6, 2017
@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

The current behavior is not great in this regard. Would prefer it to throw a fatal exception at the first attempt to set a LHS variable so the user is aware that their code must be fixed.

On second thought, a warning for LHS variable use would be better. Changing to it a fatal exception would be a breaking change unfortunately. There's probably use of mixed RHS/LHS use in the wild.

By the way it's possible to test for warnings in the test framework. See:

https://github.com/mishoo/UglifyJS2/blob/c55dd5ed74888d533cf8402c6ba3612916ba2885/test/compress/issue-1034.js#L29-L34

});
});
it("Should work with --define", function (done) {
var command = uglifyjscmd + ' test/input/issue-1467/sample.js --define D=5 -cm';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think -cm could be replaced with -c. I only put mangle in the post out of habit.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. I type faster than I think. 😅

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Incorporated those tests you've mentioned above except the first one, which is a mixed LHS/RHS case.

I shall now work on that mixed case and emit warnings as requested. 😉

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Incorporated those tests you've mentioned above except the first one, which is a mixed LHS/RHS case.

I'd like to see a mixed use LHS/RHS test case with expect_warnings: to see what happens.

Edit: I should better read your comments...

I shall now work on that mixed case and emit warnings as requested

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

alexlamsl commented Feb 6, 2017

I'd like to see a mixed use LHS/RHS test case with expect_warnings: to see what happens.

Would you like something difference from https://github.com/mishoo/UglifyJS2/pull/1467/files#diff-a304d9611b5625800249b63bea77fce7R33 ?

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

FWIW, test/benchmark.js returns identical SHA1s on master and e54fd32

function f(CONFIG) {
return CONFIG.VALUE;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please remove the empty lines in the test so it's easier to view the test in its entirety on screen at once?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

From a CLI usability point of view, this is awkward:

$ echo 'console.log(C.D)' | bin/uglifyjs -c unsafe --define C={D:true} 
console.log(!0);

The unsafe is not necessary, and the object nesting is not intuitive.

Users would expect to use it in this way:

$ echo 'console.log(C.D)' | bin/uglifyjs -c --define C.D=true 
console.log(C.D);

with it producing:

console.log(!0);

Think of the original use case of replacing process.env.NODE_ENV. Users would like to enter something like this:

$ echo 'console.log(process.env.NODE_ENV)' | bin/uglifyjs -c --define 'process.env.NODE_ENV="production"'

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

That is the use case I have in mind, though I use it via API. Your case about --define certainly makes sense - I'll look into making it happen.

unsafe is required due to #1425 - and I think we can relax that at a future release after some field testing.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

unsafe is required due to #1425 - and I think we can relax that at a future release after some field testing.

I see. Could you document that in the README? It's not an obvious association.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Could you document that in the README?

Will do 👍

With regards to global_defs, do you agree specifying:

global_defs: {
  process: {
    env: { NODE_ENV: "production" }
  }
}

to be more natural in the API form?

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

I'm not sure. Webpack apparently supports both:

new webpack.DefinePlugin({
    'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV || 'development')
}),

and

new webpack.DefinePlugin({
  "process.env": {
    NODE_ENV: JSON.stringify("production")
  }
});

Can you do something similar - assume dots in keys imply parent objects?

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

@kzc feel free to suggest anything to expand on README.md - writing documentation is not my forte 😅

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Your README addition looks fine.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

With latest PR did not see any warnings for LHS defines:

$ echo 'console.log(++D.E);' | bin/uglifyjs --define D.E=123 -c unsafe
console.log(++123);
$ echo 'console.log(++D.E);' | bin/uglifyjs --define D=123 -c unsafe
console.log(++123..E);

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Should be all fixed up now.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

(Travis CI timed out.)

@alexlamsl alexlamsl closed this Feb 6, 2017
@alexlamsl alexlamsl reopened this Feb 6, 2017
@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Just checked 84d3d5a - SHA1s from test/benchmark.js still identical with those from master.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Not directly relevant to this PR per se, but there are code out there which uses IIFEs to capture global variables:

(function(window) {
  // mian program code
})(window);

Which aids mangling, but will be opaque to global_defs for the purpose of conditional compilation.

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Warning is incorrect here in latest PR:

$ echo 'console.log(++A.B.C);' | bin/uglifyjs --define A.B.X=123 -c unsafe
WARN: global_defs["A"] redefined [-:1,14]
console.log(++A.B.C);

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Just curious how we might achieve replacing a symbol name with another symbol name...

String replacement "works" in a sense:

$ echo 'println(123);' | bin/uglifyjs --define println='"console.log"' -c
"console.log"(123);

but how can we achieve the following output?

console.log(123);

@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 6, 2017

Sorry - let's try that again...

With this PR the following does not work:

$ echo 'println(123);' | bin/uglifyjs --define println=console.log -c
println(123);
$ echo 'console.log(456);' | bin/uglifyjs --define console.log=println -c
undefined:2
return (println)
        ^

ReferenceError: println is not defined

I recognize it does not work in uglify-js@2.7.5 either. But could it be supported somehow?

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

@kzc sorry for the botched fix - please try this second attempt after some sleep.

Meanwhile I'll look at the case above.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Swapping println with console.log is actually AST_Node substitutions. While the command-line case looks simple as you demonstrated above, the API case seems a little contrived:

{
  global_defs: {
    println: UglifyJS.parse("console.log").body[0].body
  }
}

Any better ideas?

Also, /me already thinks bin/uglifyjs --define A=B,B=A is a fun thing to try 😈

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Speaking of the devil, echo a; | bin/uglifyjs -d A=console.log('PWNED!') -c gives some interesting results.

- no longer throw errors against objects
- now depends on `evaluate`
- `unsafe` when using objects
- support `"a.b":1` on both cli & API
- emit warning if variable is modified

fixes mishoo#1416
closes mishoo#1198
@alexlamsl
Copy link
Copy Markdown
Collaborator Author

@kzc so I think about it some more on this AST_Node substitution, and it's essentially a TreeTransformer walk replacing all the matching AST_SymbolRef with a supplied AST_Node, skipping over all the isLHS() instances.

Would you mind elaborating on the general use case of this feature? I'm trying to gauge the scope of work to see if this is straightforward or I should branch it to another PR.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Another thing I'm thinking about for future work is to allow functions to global_defs:

{
  global_defs: {
    require: function(path) {
      if (path == "./config") return { env: "production", isNode: false }
      throw this;
    }
  }
}

Which will also work on command line as well.

@alexlamsl alexlamsl mentioned this pull request Feb 7, 2017
@kzc
Copy link
Copy Markdown
Contributor

kzc commented Feb 7, 2017

I thought substituting functions would be a nice to have. I appreciate it's non-trivial and involves creating nodes. It's not essential.

@alexlamsl
Copy link
Copy Markdown
Collaborator Author

Closing in favour of #1469.

@alexlamsl alexlamsl closed this Feb 8, 2017
@alexlamsl alexlamsl deleted the global_defs-unsafe branch February 8, 2017 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conditional compilation problem

2 participants