Skip to content

Bootstrap Babel on npm install#5309

Closed
chicoxyzzy wants to merge 1 commit intobabel:masterfrom
chicoxyzzy:auto_bootstrap
Closed

Bootstrap Babel on npm install#5309
chicoxyzzy wants to merge 1 commit intobabel:masterfrom
chicoxyzzy:auto_bootstrap

Conversation

@chicoxyzzy
Copy link
Copy Markdown
Member

Q A
Patch: Bug Fix? no
Major: Breaking Change? no
Minor: New Feature? no
Deprecations? no
Spec Compliancy? no
Tests Added/Pass? no
Fixed Tickets Fixes #5299
License MIT
Doc PR
Dependency Changes

@mention-bot
Copy link
Copy Markdown

@chicoxyzzy, thanks for your PR! By analyzing the history of the files in this pull request, we identified @hzoo, @danez and @bcoe to be potential reviewers.

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 14, 2017

If we do this we can change the contributing.md https://github.com/babel/babel/blob/master/CONTRIBUTING.md#setup to say npm install instead of make bootstrap

although I would also add a section to explain what bootstrap does?

actually it's the same behavior, so either works

@danez
Copy link
Copy Markdown
Member

danez commented Feb 14, 2017

This would run npm install twice for the root I think, but probably is not a big issue.

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 14, 2017

Oh right - hmm I wonder what happens for yarn for 7.0

@chicoxyzzy
Copy link
Copy Markdown
Member Author

This would run npm install twice for the root

That's true but preinstall will be called once

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 14, 2017

yarn command seems to work properly

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 14, 2017

hmmm... that's weird but I see npm install call only once in my console using either npm or yarn 😃

@existentialism existentialism added the PR: Internal 🏠 A type of pull request used for our changelog categories label Feb 14, 2017
@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 14, 2017

Maybe it won't run again if it's currently being run 😄 ?

@chitchu
Copy link
Copy Markdown
Contributor

chitchu commented Feb 15, 2017

One caveat of the --ignore-scripts switch is it also prevents dependency scripts to run.

And also upgrading/adding a new dependency will run make bootstrap every time.

Also maybe you could split bootstrap by adding both preinstall and postinstall script where preinstall cleans up all node_modules folder and postinstall does the bootstrap? So the lifecycle is much cleaner as such:

  1. preinstall will clean-all
  2. install
  3. postinstall will bootstrap

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 15, 2017

I've tried this but npm install fails when I run it for second time (when there are non-empty node_modules directory already). It does clean node_modules but it doesn't actually run install script before postinstall for some reason and I'm getting following error:

make: ./node_modules/.bin/lerna: No such file or directory

related npm issue: npm/npm#10379

@chitchu
Copy link
Copy Markdown
Contributor

chitchu commented Feb 15, 2017

I think you might need to remove make clean-all from bootstrap.

Edit: nm I misread.

@chicoxyzzy
Copy link
Copy Markdown
Member Author

That's exactly what I did:

bootstrap:
	./node_modules/.bin/lerna bootstrap
	make build
	cd packages/babel-runtime; \
	npm install; \
	node scripts/build-dist.js
    "preinstall": "make clean-all",
    "postinstall": "make bootstrap"

There is an issue in npm preinstall script logic (described in issue I mentioned)

@chitchu
Copy link
Copy Markdown
Contributor

chitchu commented Feb 15, 2017

Maybe make a conditional npm install in bootstrap?

@chicoxyzzy
Copy link
Copy Markdown
Member Author

Just realized that if we'll make Babel run tests via npm install && npm test, than we'll be able to propose Babel for inclusion in Node.js CITGM runs. If Babel will be included in CITGM than each new version of Node.js will be smoke-tested using Babel among other npm modules.

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 15, 2017

That would cool! cc @MylesBorins

@MylesBorins
Copy link
Copy Markdown
Member

As long as releases continue to use a consistent tagging as I'm seeing in https://github.com/babel/babel/tags we could definitely get babel on CITGM!

Does someone want a good first contrib to www.github.com/nodejs/citgm ??

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 15, 2017

Might be a good beginner-friendly issue?

The test process must be executable with only the commands npm install && npm test using the tarball downloaded from the Github tag mentioned above

So it should be ok if we use preinstall then?

@MylesBorins
Copy link
Copy Markdown
Member

@hzoo preinstall should be fine as long as you are not modifying the env

@chicoxyzzy
Copy link
Copy Markdown
Member Author

I'm getting following error when running CITGM

➜  citgm git:(master) ✗ ./bin/citgm.js -v verbose babel
info:    starting            | babel
verbose: babel using-uid     | 501
verbose: babel using-gid     | 20
verbose: babel using-node    | /Users/Chico/.nvm/versions/node/v7.5.0/bin/node
verbose: babel using-npm     | /Users/Chico/.nvm/versions/node/v7.5.0/bin/npm
verbose: babel mk.tempdir    | /var/folders/rt/rl8_2jsn67l24mvmfwdkgqzm0000gn/T/6c4015d7-060b-43c4-93db-97509d257119
info:    lookup              | babel
info:    lookup-found        | babel
info:    babel lookup-replace| https://github.com/babel/babel/tree/master/packages/babel/archive/v6.23.0.tar.gz
info:    babel npm:          | Downloading project: https://github.com/babel/babel/tree/master/packages/babel/archive/v6.23.0.tar.gz
error:   failure             | Failure getting project from npm
error:   failing module(s)   |
error:   module name:        | babel
error:   version:            | 6.23.0
error:   error:              | Failure getting project from npm
error:   error:              |
error:   done                | The smoke test has failed.
info:    duration            | test duration: 73385ms

lookup.json

  "babel": {
    "prefix": "v"
  }

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 15, 2017

Is it because Babel is a monorepo?

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 15, 2017

Adding "https://github.com/babel/babel" as value of repo field makes "Project link" correct (omits /tree/master/packages/babel part of "Downloading project" url, now it's downloadable) but I'm still getting same error

@MylesBorins
Copy link
Copy Markdown
Member

MylesBorins commented Feb 15, 2017

So we have run into this edge case before and it is caused by npm pack blowing up on the tarball.

we can repro with citgm via npm pack https://github.com/babel/babel/archive/v6.23.0.tar.gz

The first time we ran into it was with lerna usage on pug

--> npm/npm#15563

I'm digging into the errors from npm to see what's up, this is a slightly different edge case. Will update inline

edit: it is because the package.json inside the babel tarball does not have a name. Is there a reason you publish with a stripped down package.json?

@xtuc
Copy link
Copy Markdown
Member

xtuc commented Feb 15, 2017

I think this should only run the first time you clone Babel. Each time we add a new dependecy and run npm install it will cause to compile Babel.

@chicoxyzzy
Copy link
Copy Markdown
Member Author

As far as I understand this is possible only using post-checkout GIT hook. But CITGM will ignore any GIT hooks

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 15, 2017

edit: it is because the package.json inside the babel tarball does not have a name. Is there a reason you publish with a stripped down package.json?

No reason it's just a monorepo and the top level package.json isn't a package, but holds our devDeps so probably didn't think it was necessary to have a name (and we have a babel package at packages/babel)

but #5311 should fix!

@chicoxyzzy
Copy link
Copy Markdown
Member Author

Now npm pack http://github.com/babel/babel/archive/master.tar.gz fails with npm ERR! No version provided so we should have a version in package.json

@chicoxyzzy
Copy link
Copy Markdown
Member Author

can we solve this using lerna and use version from babel-core?

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 24, 2017

I have a lot of questions:

  • I'm not familiar with lerna but I've found that lerna.json has version field. Is it number of version we need to copy to main repo's package.json? npm pack needs it.
  • Is it possible to do this using lerna?
  • If not, should I contribute to lerna by adding an option to update version field in package.json?
  • If I should contribute to lerna, does that blocks this PR?
  • Should I do something else not related to CITGM in this PR?

@hzoo
Copy link
Copy Markdown
Member

hzoo commented Feb 24, 2017

If we just need a version in the top level package.json we can just add it with like 0.0.0 or something

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 24, 2017

npm pack works with "version": "0.0.0" but something goes wrong after make bootstrap, it fails with

fatal: Not a git repository (or any of the parent directories): .git

when I run ./bin/citgm.js ../babel/babel-0.0.0.tgz

@chicoxyzzy
Copy link
Copy Markdown
Member Author

chicoxyzzy commented Feb 24, 2017

It seems that lerna has more blockers for including in CITGM than just npm/npm#15563

@MylesBorins
Copy link
Copy Markdown
Member

MylesBorins commented Feb 24, 2017

if you are going to add a version I do humbly request you don't just opt for 0.0.0. We use those version number when investigating

@nicolo-ribaudo
Copy link
Copy Markdown
Member

I'm working on upgrading to yarn 2 + workspaces, which make lerna bootstrap obsolete.

@github-actions github-actions bot added the outdated A closed issue/PR that is archived due to age. Recommended to make a new issue label Apr 20, 2020
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

outdated A closed issue/PR that is archived due to age. Recommended to make a new issue PR: Internal 🏠 A type of pull request used for our changelog categories

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Better setup

9 participants