sbt-based PR validation job by szeiger · Pull Request #4884 · scala/scala

szeiger · 2015-12-16T18:43:32Z

Testing the sbt-based PR validation...

szeiger · 2015-12-16T18:58:45Z

@SethTisue I think if you add the publishing stuff to the script that we figured out on Friday, it should actually work. Can you try that?

Remaining issues: Enable optimization & do not publish docs

SethTisue · 2015-12-16T19:48:02Z

commit added, let's see what happens

SethTisue · 2015-12-16T19:55:38Z

looks like publish-core went fine, but then of course validate-test couldn't run because we aren't yet publishing to https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/

SethTisue · 2015-12-16T22:08:27Z

getting closer:

[error] Unable to find credentials for [Artifactory Realm @ scala-ci.typesafe.com].

SethTisue · 2015-12-16T22:22:16Z

yay a bunch of JARs actually got published to the scala-ci artifactory 🎉

but then it bombed with

java.lang.RuntimeException: Repository for publishing is not specified.
    at scala.sys.package$.error(package.scala:27)
    at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:1493)
    at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:1493)
    at scala.Option.getOrElse(Option.scala:120)
    at sbt.Classpaths$.getPublishTo(Defaults.scala:1493)
    at sbt.Classpaths$$anonfun$62.apply(Defaults.scala:1165)
    at sbt.Classpaths$$anonfun$62.apply(Defaults.scala:1165)
    at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$$anon$4.work(System.scala:63)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
[error] (root/*:publishConfiguration) Repository for publishing is not specified.

EDIT: this is even though the root project has publishArtifact := false

SethTisue · 2015-12-16T23:03:03Z

hmm so artifacts published during https://scala-ci.typesafe.com/job/scala-2.11.x-validate-publish-core/2244/console showed up at e.g. https://scala-ci.typesafe.com/artifactory/scala-pr-validation-snapshots/org/scala-lang/scala-library/2.11.8-f0d538a-SNAPSHOT/ but then https://scala-ci.typesafe.com/job/scala-2.11.x-validate-test/1609/console didn't find all (any?) of the artifacts it wanted

SethTisue · 2015-12-16T23:07:24Z

perhaps the problem is the mix of -1 and -2 suffixes (here's ant not doing that). but what is causing those suffixes?

SethTisue · 2015-12-16T23:32:51Z

ah, this appears to be sbt/sbt#2088

adriaanm · 2015-12-17T00:31:57Z

can we just drop the -SNAPSHOT?

szeiger · 2016-01-07T15:56:13Z

Dropping -SNAPSHOT would mean that we'd have to change the existing infrastructure that relies on these version numbers. It's the first time I've seen these "-1" and "-2" suffixes. I have no idea what they are and where they come from. I'll have to check the sbt sources.

szeiger · 2016-01-07T17:19:12Z

build.sbt

@SethTisue What is the reason for using this over publishTo? It is only referenced in one place (for setting publishTo)

szeiger · 2016-01-08T11:33:10Z

/rebuild

szeiger · 2016-01-11T12:36:25Z

As far as I can tell this looks good. The integrate-ide job (and consequently validate-main) has been failing for several weeks. @SethTisue WDYT? Shall I clean up the code and squash?

(Still looking into the optimization and docs stuff)

szeiger · 2016-01-11T17:40:20Z

Docs disabled, optimization enabled. The doc artifacts are gone and validate-test passes, so this should be ready for squashing and merging.

adriaanm · 2016-01-11T19:07:49Z

🍰!

# Improve version handling in the sbt build: The new settings `baseVersion` and `baseVersionSuffix` make it easier to set version numbers for different kinds of builds in a consistent way without e.g. having to first get a git SHA outside of sbt. The new task `generateBuildCharacterPropertiesFile` writes the file `buildcharacter.properties` to the root dir. The format is compatible with the same file produced by the ANT build but it only contains a subset of the properties, in particular the Maven version, which is needed in publishing scripts and passed around between different Jenkins jobs as `jenkins.properties`. SHAs in version numbers are consistently shortened to 7 digits (as used by git and github). Previously we used 7 digits in Maven snapshot version numbers but 10 digits in canonical and OSGi version numbers. # Add Jenkins script support to the sbt build: The new command `setupPublishCore` takes the PR validation snapshot repository as an argument and changes the required settings for the `publish-core` build (use SHA-SNAPSHOT versioning, compile with optimization enabled, do not publish scaladoc sets) For example, the following command can be used to generate `buildcharacter.properties` with the version numbers needed for PR validation builds: sbt setupPublishCore dummy generateBuildCharacterPropertiesFile The sbt build will now automatically detect and use a “~/.credentials” file with the credentials for publishing to a remote repository. # Call sbt from `publish-core`: The correct`$SBT_CMD` is set directly in `bootstrap` and used by `publish-core` to first generate `buildcharacter.properties` and then build and publish. Parsing the git revision, computing a version number and getting binary dependencies are no longer required in the script. This is all done in the sbt build.

szeiger · 2016-01-12T13:19:51Z

Cleaned up and squashed.

SethTisue · 2016-01-13T21:26:00Z

/nothingtoseehere

(integrate-ide failure is unrelated to the PR)

dwijnand · 2016-01-15T17:40:22Z

What was done to avoid sbt/sbt#2088?

szeiger · 2016-01-18T12:40:58Z

@dwijnand Setting a consistent build.timestamp: https://github.com/szeiger/scala/blob/wip/sbt-pr-validation/project/ScriptCommands.scala#L14

SethTisue · 2016-01-25T19:18:21Z

@szeiger may I suggest some light testing with sbt 0.13.10-RC1? it'd be a bit awkward if we merged this and then were immediately stuck on 0.13.9 because of some regression

szeiger · 2016-01-26T11:44:43Z

@SethTisue No luck with 0.13.10-RC1: https://gist.github.com/szeiger/d494e70da91eb1ec9d06

dwijnand · 2016-01-26T12:13:32Z

You want to try 0.13.10-M1 (but @eed3si9n might want to cut a new release).

Or you might want to wait for a successful community build for 0.13.10, see scala/community-build#210.

szeiger · 2016-01-26T14:33:48Z

I was able to build it successfully with sbt 0.13.10-M1 but not with 0.13.10-RC1. /cc @eed3si9n

dwijnand · 2016-01-26T14:35:00Z

That's good news. Thank you @szeiger.

sbt-based PR validation job

adriaanm · 2016-01-29T21:31:50Z

time to merge 2.11.x into 2.12.x?

dwijnand · 2016-01-29T21:32:04Z

Cool!

SethTisue · 2016-01-29T21:35:08Z

time to merge 2.11.x into 2.12.x?

we're not that close yet. this PR makes validate-publish-core sbt-based, but doesn't update validate-test (not to mention integrate-bootstrap and integrate-windows). we can discuss overall status at scala/scala-dev#45

this is the next step after scala#4884

rather use sbt and direct invocation of partest this is the next Ant-elimination step after scala#4884

scala-jenkins added this to the 2.11.8 milestone Dec 16, 2015

szeiger reviewed Jan 7, 2016
View reviewed changes

szeiger force-pushed the wip/sbt-pr-validation branch from 17c450e to b1b5475 Compare January 12, 2016 13:19

SethTisue self-assigned this Jan 13, 2016

szeiger mentioned this pull request Jan 25, 2016

Some fixes for the sbt build on Windows #4913

Merged

SethTisue added a commit that referenced this pull request Jan 29, 2016

Merge pull request #4884 from szeiger/wip/sbt-pr-validation

3c26633

sbt-based PR validation job

SethTisue merged commit 3c26633 into scala:2.11.x Jan 29, 2016

This was referenced Jan 29, 2016

tweak my own entry in the readme #4932

Merged

finish the sbt build scala/scala-dev#45

Closed

SethTisue added a commit to SethTisue/scala that referenced this pull request Jan 29, 2016

when running tests during PR validation, use sbt not ant

aef30bb

this is the next step after scala#4884

SethTisue mentioned this pull request Jan 29, 2016

when running tests during PR validation, use sbt not ant [ci: last-only] #4933

Closed

SethTisue added a commit to SethTisue/scala that referenced this pull request Jan 29, 2016

when running tests during PR validation, use sbt not ant

d91ee73

this is the next step after scala#4884

SethTisue added a commit to SethTisue/scala that referenced this pull request Feb 3, 2016

when running tests during PR validation, don't use ant

93b84a3

rather use sbt and direct invocation of partest this is the next Ant-elimination step after scala#4884

Conversation

szeiger commented Dec 16, 2015

Uh oh!

szeiger commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

SethTisue commented Dec 16, 2015

Uh oh!

adriaanm commented Dec 17, 2015

Uh oh!

szeiger commented Jan 7, 2016

Uh oh!

szeiger Jan 7, 2016

Choose a reason for hiding this comment

Uh oh!

szeiger commented Jan 8, 2016

Uh oh!

szeiger commented Jan 11, 2016

Uh oh!

szeiger commented Jan 11, 2016

Uh oh!

adriaanm commented Jan 11, 2016

Uh oh!

szeiger commented Jan 12, 2016

Uh oh!

SethTisue commented Jan 13, 2016

Uh oh!

dwijnand commented Jan 15, 2016

Uh oh!

szeiger commented Jan 18, 2016

Uh oh!

SethTisue commented Jan 25, 2016

Uh oh!

szeiger commented Jan 26, 2016

Uh oh!

dwijnand commented Jan 26, 2016

Uh oh!

szeiger commented Jan 26, 2016

Uh oh!

dwijnand commented Jan 26, 2016

Uh oh!

adriaanm commented Jan 29, 2016

Uh oh!

dwijnand commented Jan 29, 2016

Uh oh!

SethTisue commented Jan 29, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants