Skip to content

[WIP] Migrate from Gulp to Fly#965

Merged
timneutkens merged 19 commits intovercel:masterfrom
lukeed:fly
Feb 9, 2017
Merged

[WIP] Migrate from Gulp to Fly#965
timneutkens merged 19 commits intovercel:masterfrom
lukeed:fly

Conversation

@lukeed
Copy link
Copy Markdown
Contributor

@lukeed lukeed commented Feb 3, 2017

Fly has finally reached 2.0, which is super lightweight and brings speed(!) and efficient memory usage. Not too long ago, it was mentioned (#435 (comment)) that Next.js may want to migrate towards Fly because of these reasons.

The current master is failing during Jest testing. Since I've dealt purely with Gulp → Fly, it will still fail.

To Do

  • Add fly-esnext
  • The bench task is currently broken. Once it's been fixed & I see how it's meant to be, I will port that task, too.
  • Cleanup the build-prefetcher webpack-based task. I am working on a PR for fly-webpack

Thoughts

  • Extract the lint command into a lint task; NPM script would then contain: fly lint

  • Possibly change some task names? Using non-hyphenated names would allow for a more consistent & cleaner-looking flyfile:

    exports.build = function * (fly) {}
    exports.watch = function * (fly) {}
    // or
    module.exports = {
      * build(fly) {},
      * watch(fly) {}
    }
  • Does Next.js use the root .babelrc to run everything?

    If not, move it to config/babel.js & webpack.config.js to config/webpack.js.

Improvements

Gulp Fly
Total Dependencies 797 718
Time: prepublish 13s 8.5s
Time: release 5.8s 4.8s
Memory: release 145MB 133MB

If merged, closes #884.

@rauchg
Copy link
Copy Markdown
Member

rauchg commented Feb 3, 2017

This is fantaaaaaaaastic

@rauchg
Copy link
Copy Markdown
Member

rauchg commented Feb 3, 2017

Btw feel free to ditch the hyphens

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

Here's screenshot of npm run release:

screen shot 2017-02-02 at 3 28 38 pm

And a screenshot of running build with a watcher trigger:

screen shot 2017-02-02 at 5 44 56 pm

Btw feel free to ditch the hyphens

Since every team has their own naming conventions, I don't want to step on anybody's toes & rename them in a way that makes less sense for the more active contributors.

Also edited original post with a todo re: lint task.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

Lastly, I know that ZEIT prefers async/await. So I can setup fly-esnext to allow for async task definitions.

It doesn't really add anything except for visual appeal...

export async function build(fly) {
  await fly.source('src/*.js')...
}

export default async function(fly) {
  await fly.start('build')
  await fly.watch('...')
}

@nkzawa
Copy link
Copy Markdown
Contributor

nkzawa commented Feb 3, 2017

I prefer async/await and hope fly supports it as default 😍

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

@nkzawa I was playing with this locally. It can definitely be done, but it's a matter of trading dependencies & speed for syntactical preferences.

My gut says that opting in to a plugin is the right choice.

@nkzawa
Copy link
Copy Markdown
Contributor

nkzawa commented Feb 3, 2017

@lukeed Is the speed issue because of transpilation ? I think it can be solved since native async/await would be introduced on Node soon, or by using https://github.com/leebyron/async-to-gen

@arunoda
Copy link
Copy Markdown
Contributor

arunoda commented Feb 3, 2017

@lukeed I'm also in favor of async/await. Generators feels like bit of a magic.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

@nkzawa Fly is powered by Bluebird; more specifically, Bluebird's coroutine implementation. This is because it's consistently the fastest Promise/Promise-alternative there is, for now. It's also the most memory efficient -- I spent weeks testing over 40 libraries. It's the clear winner.

Right now, the current game plan is reevaluate all benchmarks once async is supported natively. If and only if the native implementation can contend with Bluebird's performance will I consider adopting a pure native approach.

Fly isn't meant to be a prime example of what can be done with Node built-ins. Admittedly, it'd be awesome! I love that about most ZEIT products. But the only way Fly can even be considered over Gulp or Grunt is if it's considerably & unequivocally faster.

That said, I had a fork with async-to-gen, but it applies its own wrapper around the native Promise class. Even after some tinkering, I could get it to write Promises with Bluebird.promise, at best, but those are still slower than Bluebird.coroutine. 😞

This is why I made fly-esnext. All it does is rewrite an async function into a generator function (behind the scenes) & then evaluates the "transpiled" flyfile. And since flyfile.js is the only point of contact from the User and Fly, that's the only thing that needs rewriting. The plugin hides all generators from the user (no more magic for @arunoda), but Fly still receives the generators it needs, then wraps 'em into coroutines.

We can continue this discussion at a later time (very interested in your feedbacks!) but installing fly-esnext for this PR is the quickest & simplest solution.

@arunoda
Copy link
Copy Markdown
Contributor

arunoda commented Feb 3, 2017

@lukeed nice.
Shall we use fly-esnext to this?
(I hope that's possible)

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

@arunoda You bet. I'll add it tomorrow. I've updated my todo list with fly-esnext as a new item.

@jorgebucaran
Copy link
Copy Markdown

jorgebucaran commented Feb 3, 2017

@nkzawa Fly used to support async/await, mind you, that was before Babel 6. It was slow. So, we removed it prior to 1.0.

@arunoda Generators feels like bit of a magic.

Generators feel more lower level, if anything. While I do find async/await better looking than function* myself, introducing fly-esnext in a later PR seems more sensible.

Don't you agree @lukeed?

@sedubois
Copy link
Copy Markdown
Contributor

sedubois commented Feb 3, 2017

Would using fast-async change things somehow?

#851

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

@sedubois Unfortunately, no. You still suffer Babel's 350-600ms boot time, and the end result is still slower & less efficient. Bluebird is hands-down the fastest option.

Having the ability to write async/await tasks in Fly is not a problem, guys. This is what fly-esnext solves. It's already doable & with near-zero perf loss.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 3, 2017

Sorry @jbucaran, missed your comment. I think we should just include fly-esnext in here. The guys here all want it & it's not our project anyway 😉 Plus, the performance is the same; there's only an increase in memory usage...but it's still lower than Gulp!

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 5, 2017

Rebased & forced update. PR now includes the async/await version. @arunoda @nkzawa @rauchg

Task names were also truncated; eg compile-lib --> lib, remove-strict-mode --> unrestrict. If it were my personal project, I'd just replace the hyphens with underscores. ¯_(ツ)_/¯

Stat improvements are the same.

PR is still failing because master fails; and this is still lacking a full bench task.

@nkzawa
Copy link
Copy Markdown
Contributor

nkzawa commented Feb 5, 2017

@lukeed it looks master's CI succeeds.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 5, 2017

@nkzawa Ah, I just inspected. Did not realize that Next.js is testing Node 4.x...

Fly only supports >= 6 because 4 is dropping LTS in April.

@arunoda
Copy link
Copy Markdown
Contributor

arunoda commented Feb 5, 2017

@lukeed But we need to support Node 4 even after that due to legacy reasons.
Some people still might use it.

But anyway, we need to support it for at-least until April.

@rauchg
Copy link
Copy Markdown
Member

rauchg commented Feb 5, 2017 via email

@rauchg
Copy link
Copy Markdown
Member

rauchg commented Feb 5, 2017 via email

@arunoda
Copy link
Copy Markdown
Contributor

arunoda commented Feb 5, 2017

@rauchg If we could that's nice.
But in order to test them in travis, we need to build it on travis.

If not, we could change the travis build tool to handle Node versions manually.
Always, build in latest node and change the node version to do the test.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 5, 2017

Up to you guys. Fly originally supported 4.x, but switched to 6.x at the last minute. All we got out of it was a couple spread operators & destructured assignments.

@rauchg
Copy link
Copy Markdown
Member

rauchg commented Feb 5, 2017 via email

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 5, 2017

Yeah, I'll patch it. I like that rule too; just took a shortcut this time around.

cc @jbucaran, this was my hesitation for dropping support for 4.

const processName = isWindows ? 'chromedriver.cmd' : 'chromedriver'
const chromedriver = childProcess.spawn(processName, { stdio: 'inherit' })
// We need to do this, otherwise this task's process will keep waiting.
setTimeout(() => process.exit(0), 2000)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's up w/ this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per the comment above, they need to do this to get chromedriver to shut down at the proper time. Ported directly from the gulpfile.

@jorgebucaran
Copy link
Copy Markdown

@lukeed Right!

BTW which 4.x?

=4.6 would be best.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 5, 2017

Is there something that could be affecting the runtime of NPM scripts? Travis is now throwing because of a "reserved word" (require). However, I just reproduced a simple project that only uses the Fly plugins here & the same error does not appear on 4.7.

Any ideas?

@hzlmn
Copy link
Copy Markdown

hzlmn commented Feb 5, 2017

@lukeed Maybe problem with Travis caching. It seems like it not re-install modules.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 9, 2017

@timneutkens Grabbed all of the latest~

Copy link
Copy Markdown
Member

@timneutkens timneutkens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lukeed Thanks ❤️ Side note on this: I really like the async/await syntax combined with fly 😍

@timneutkens timneutkens merged commit 839fb1c into vercel:master Feb 9, 2017
@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 9, 2017

Thanks 😄 Hopefully, one day, I can just drop Fly into native Node 8(?) & it'll just work. Crossing my fingers for massive boost in native Promise performance. 🤞

@timneutkens
Copy link
Copy Markdown
Member

timneutkens commented Feb 9, 2017

@lukeed found a small bug, the binaries are copied over to dist/bin the thing I'm seeing is that they lose the given rights they had (execution). Which breaks next.

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 9, 2017

@timneutkens Thanks! Fix incoming.

@timneutkens
Copy link
Copy Markdown
Member

❤️

@lukeed
Copy link
Copy Markdown
Contributor Author

lukeed commented Feb 9, 2017

@timneutkens Should I delete the gulpfile.js in my next PR, or leave it for reference on that bench task?

@lukeed lukeed deleted the fly branch February 9, 2017 20:40
@lock lock bot locked as resolved and limited conversation to collaborators Jan 19, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants