`make dist` for creating binaries and packaging them for distribution by melekes · Pull Request #378 · tendermint/tendermint

melekes · 2017-01-23T11:41:28Z

Preface: if we want to engage more people (not just hardcore devs), I believe we must provide statically compiled binaries for each release. Besides, this could be easily done with Golang.

Story: as a dev, if I just want to try Tendermint (or I am planning to write my app in Java), I don't want to install Golang, I just want brew install tendermint or similar (btw, what do you think about adding Tendermint to brew).

This will fix many issues (https://tendermint.com/intro/getting-started/install):

user don't have to install Golang
user don't have to learn Golang tools (go get)
user won't be having any problems with updated dependencies

Usage

$ make dist

will create binaries for almost all platforms and will package them for distribution. It will also create a tag (e.g. "v0.8.0").

If you've already created a tag, set NOTAG env variable to true:

$ NOTAG=1 make dist

RELEASE could be set explicitly via env variable. Otherwise, the script will try to fetch it from version/version.go

Broken builds

Our code currently does not compile for freebsd/amd64:

--> freebsd/amd64 error: exit status 2
Stderr: # github.com/tendermint/tendermint/vendor/github.com/spf13/pflag
vendor/github.com/spf13/pflag/uint16.go:89: illegal NUL byte
vendor/github.com/spf13/pflag/uint16.go:89: syntax error: illegal character U+0000

and solaris/amd64:

--> solaris/amd64 error: exit status 1
Stderr: vendor/github.com/tendermint/log15/root.go:7:2: no buildable Go source filesin /go/src/github.com/tendermint/tendermint/vendor/github.com/tendermint/log15/term

Example output

==> Results:
total 49M
-rw-r--r-- 1 vagrant vagrant 5.4M Jan 23 11:12 tendermint_0.8.0_darwin_386.zip
-rw-r--r-- 1 vagrant vagrant 5.7M Jan 23 11:12 tendermint_0.8.0_darwin_amd64.zip
-rw-r--r-- 1 vagrant vagrant 6.1K Jan 23 10:51 tendermint_0.8.0_.DS_Store
-rw-r--r-- 1 vagrant vagrant 5.4M Jan 23 11:12 tendermint_0.8.0_freebsd_386.zip
-rw-r--r-- 1 vagrant vagrant 5.2M Jan 23 11:12 tendermint_0.8.0_freebsd_arm.zip
-rw-r--r-- 1 vagrant vagrant 5.4M Jan 23 11:12 tendermint_0.8.0_linux_386.zip
-rw-r--r-- 1 vagrant vagrant 5.6M Jan 23 11:12 tendermint_0.8.0_linux_amd64.zip
-rw-r--r-- 1 vagrant vagrant 5.2M Jan 23 11:12 tendermint_0.8.0_linux_arm.zip
-rw-r--r-- 1 vagrant vagrant 1002 Jan 23 11:12 tendermint_0.8.0_SHA256SUMS
-rw-r--r-- 1 vagrant vagrant 5.4M Jan 23 11:12 tendermint_0.8.0_windows_386.zip
-rw-r--r-- 1 vagrant vagrant 5.6M Jan 23 11:12 tendermint_0.8.0_windows_amd64.zip

What could be done

sign tag with GPG signature (easily done)
sign tendermint_VERSION_SHA256SUMS with GPG signature (easily done)
figure out why our code does not compile for freebsd and solaris, and fix it

Inspiration: https://www.hashicorp.com/security.html

- mute most of the commands - replace github.com/tendermint/tendermint with just "." - introduce PACKAGES variable - delete unused NEWLINE

melekes · 2017-01-23T11:46:39Z

version/version.go

 package version

 const Maj = "0"
-const Min = "8" // validator set changes, tmsp->abci, app persistence/recovery, BFT-liveness fix


Is this the best place for such comments? maybe we should create CHANGELOG.md instead?

I've been keeping the changelog in the release notes on github for every release. Not opposed to also adding them all into a CHANGELOG.md. the comment here is just a rough reference if you're cruising by

ethanfrey · 2017-01-23T18:40:12Z

I just saw this issue. Yes, I love the idea!

Re outstanding points 1 and 2.

I think the gpg signing should happen outside of the make dist. It can just build stuff, if we want signing, this is a special key kept on a special machine, so we shouldn't make that part of dist, which I may want to do on my own without signing. It would be great to have a second command to do that, maybe make signed-dist, which calls make dist and then gpg signs using the location of a key set in an environment variable???

GPG_KEY=/home/foo/.gpg/tm.key make signed-dist ?

The other question... can we check all of them behave properly to a basic test? I know there are some issues with cgo when cross-compiling, maybe you dealt with that, maybe it is not an issue,but it would be nice to have an integration test.

Also, that is some crazy shell scripting skills. Props!

melekes · 2017-01-23T18:57:24Z

I think the gpg signing should happen outside of the make dist. It can just build stuff, if we want signing, this is a special key kept on a special machine, so we shouldn't make that part of dist, which I may want to do on my own without signing.

or we can have a flag (NOSIGN)

The other question... can we check all of them behave properly to a basic test? I know there are some issues with cgo when cross-compiling, maybe you dealt with that, maybe it is not an issue,but it would be nice to have an integration test.

interesting question! for now, I just disabled cgo (do we have C libraries?) 👿 I have a second laptop with Linux, so I will definitely check that

ethanfrey · 2017-01-23T19:01:50Z

I think this command would be used more often without signing than with signing. At least pre-1.0. If I want to deploy a develop version onto my testnet, I would like to make dist and then scp. I could add something like that to the (advanced) tutorials as well. But maybe longer term, the signing is the more common use case.

Either way is fine actually. Just aesthetics.

I would also like to add a similar make dist command to basecoin, so that can also be deployed and hosted easily, if it is going to be our ultimate prototype app. Maybe you can help me with that, or just do this, once this PR is finalized and merged? Whichever is easier for you.

melekes · 2017-01-23T19:08:36Z

If I want to deploy a develop version onto my testnet, I would like to make dist and then scp.

hm.. we have make build, which will build a single binary. You can change the desired arch/os by using GOOS and GOARCH env variables: GOOS=windows GOARCH=amd64 make build if cross-compilation is what you want

ethanfrey · 2017-01-23T19:09:43Z

Okay, then make dist for official releases, and document how to use make build with the flags. Sounds like a good solution.

* build(deps): bump docker/login-action from 1.10.0 to 1.11.0 Bumps [docker/login-action](https://github.com/docker/login-action) from 1.10.0 to 1.11.0. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@v1.10.0...v1.11.0) --- updated-dependencies: - dependency-name: docker/login-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Run the omnibus linter always. Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: M. J. Fromberger <michael.j.fromberger@gmail.com>

…mint#380) * ci: Remove Buf tokens from workflow (tendermint#378) * ci: Remove Buf tokens from workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> * Format proto folder readme to trigger workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> (cherry picked from commit 126f190) # Conflicts: # proto/README.md * Fix conflict in proto readme Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> Co-authored-by: Thane Thomson <connect@thanethomson.com>

* ci: Remove Buf tokens from workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> * Format proto folder readme to trigger workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com>

* ci: Remove Buf tokens from workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> * Format proto folder readme to trigger workflow Signed-off-by: Thane Thomson <connect@thanethomson.com> --------- Signed-off-by: Thane Thomson <connect@thanethomson.com> (cherry picked from commit 126f190) Co-authored-by: Thane Thomson <connect@thanethomson.com>

melekes added 4 commits January 22, 2017 18:00

add .editorconfig to maintain consistent coding style

e825d77

refactor Makefile

c6375e4

- mute most of the commands - replace github.com/tendermint/tendermint with just "." - introduce PACKAGES variable - delete unused NEWLINE

dist command to make binaries and package them for distribution

cc2457f

try to get version from version/version.go

a388ff1

melekes commented Jan 23, 2017

View reviewed changes

ebuchman merged commit a388ff1 into tendermint:develop Jan 28, 2017

ebuchman mentioned this pull request Feb 14, 2017

Release v0.9.0 #401

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`make dist` for creating binaries and packaging them for distribution#378

`make dist` for creating binaries and packaging them for distribution#378
ebuchman merged 4 commits intotendermint:developfrom
melekes:feature/binaries

melekes commented Jan 23, 2017 •

edited

Loading

Uh oh!

melekes Jan 23, 2017

Uh oh!

ebuchman Jan 23, 2017

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

melekes commented Jan 23, 2017 •

edited

Loading

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

melekes commented Jan 23, 2017 •

edited

Loading

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

melekes commented Jan 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Usage

Broken builds

Example output

What could be done

Uh oh!

melekes Jan 23, 2017

Choose a reason for hiding this comment

Uh oh!

ebuchman Jan 23, 2017

Choose a reason for hiding this comment

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

melekes commented Jan 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

melekes commented Jan 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ethanfrey commented Jan 23, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

melekes commented Jan 23, 2017 •

edited

Loading

melekes commented Jan 23, 2017 •

edited

Loading

melekes commented Jan 23, 2017 •

edited

Loading