Skip to content

[sbt 1.0] sbt.Build => sbt.internal.BuildDef#2524

Merged
eed3si9n merged 70 commits intosbt:1.0.xfrom
eed3si9n:wip/buildscala
Mar 29, 2016
Merged

[sbt 1.0] sbt.Build => sbt.internal.BuildDef#2524
eed3si9n merged 70 commits intosbt:1.0.xfrom
eed3si9n:wip/buildscala

Conversation

@eed3si9n
Copy link
Member

This change removes sbt.Build from the common build classpath there by removing Build.scala dialect of the build definition. Most scripted tests are ported over to build.sbt.

before (a typical project/Build.scala)

import sbt._
import Keys._
import Import._

object build extends Build {
    override def settings = super.settings ++ Seq(scalaVersion := "2.11.7")

    val defaultSettings = Seq(
        libraryDependencies <+= scalaVersion("org.scala-lang" % "scala-reflect" % _ )
    )

    lazy val root = Project(
       base = file("."),
       id = "macro",
       aggregate = Seq(macroProvider, macroClient),
       settings = Defaults.defaultSettings ++ defaultSettings
    )

    lazy val macroProvider = Project(
       base = file("macro-provider"),
       id = "macro-provider",
       settings = Defaults.defaultSettings ++ defaultSettings
    )

    lazy val macroClient = Project(
       base = file("macro-client"),
       id = "macro-client",
       dependencies = Seq(macroProvider),
       settings = Defaults.defaultSettings ++ defaultSettings
    )
}

after (sbt 0.13 multi-project build.sbt)

lazy val commonSettings = Seq(
  libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value
)

lazy val root = (project in file(".")).
  aggregate(macroProvider, macroClient).
  settings(inThisBuild(List(
      scalaVersion := "2.11.7"
    )),
    commonSettings
  )

lazy val macroProvider = (project in file("macro-provider")).
  settings(
    commonSettings
  )

lazy val macroClient = (project in file("macro-client")).
  dependsOn(macroProvider).
  settings(
    commonSettings
  )

note

As you can see, project/Build.scala and build.sbt are not that different.

The build users can still define classes and objects in project/*.scala for instance to organize dependencies. I'm only deprecating the Build.scala style of build definitions.

/review @dwijnand, @jsuereth, @Duhemm

@dwijnand
Copy link
Member

I'm in the process of upgrading the Scala.js build, and it wasn't hard.

But it would be great if someone would create an automated tool for translation to speed up the process. 👍

@farmdawgnation
Copy link

we've hinted this for over a year.

I haven't seen anyone address this particular remark yet so I feel that I should.

Hinting at a change at a conference is not a satisfactory way of letting your community know that things are changing significantly. Not all of us go to those conferences. Those that do may not attend a session where this is hinted at. Of those that do probably only a percentage catch the actual hint and fully internalize it. If you've truly known about this for a year, sbt 0.13.x should have started surfacing a deprecation warning when it was building a Build.scala project.

Sorry if this comes off a bit rude, but I'm really quite annoyed that hasn't been done anytime in the last year as far as I can tell.

EDIT: Just realized one of those links wasn't a conference but a ML post. Which... okay that's more reasonable. But I do still feel there should have been a deprecation warning.

@eed3si9n
Copy link
Member Author

@farmdawgnation You're totally right about deprecation. The next sbt 0.13.x will have the warning (#2530), and it will be some time until sbt 1.0.x comes out.

@muuki88
Copy link
Contributor

muuki88 commented Apr 1, 2016

I like the decision. AutoPlugins where the first step to separate the how / what things are done.
build.sbt files are nice for definitions and simple settings. Your "build-logic" can go in separate plugins.

We introduced this in our company and people where starting to actually like sbt, contributing to our company wide autoplugins, and reducing the build definitions in our projects from multipe build.sbt files and Build.scala to a single, very concise build.sbt.

A migration is straightforward. You put your logic and default settings into autoplugins (e.g. http calls to jenkins for latest versions, nexus/artifactory credentials, and so on). Then you create a build.sbt where you should only use something like someSetting := "some value" or someTask := specialTask.value. Only definitions.

Great job @eed3si9n / @dwijnand and the rest of the sbt community :)

@acjay
Copy link

acjay commented Apr 3, 2016

I really feel this is the wrong direction. How is it that Scala, in its pure form, is apparently well suited to do everything, aside from describing its own builds? The .sbt layer is too clever by half, and it's a shame to see the team doubling down on it. I don't think I'm being hyperbolic by saying that this is the sort of thing that propagates the reputation of the language and ecosystem not being accessible.

Imagine you've been really sold on the benefits of Scala, and you want to use it on a new service you're writing. SBT is more or less mandatory to anybody starting a serious Scala project, and it's really difficult to avoid immediately being confronted by a whole new world of unfamiliar concepts and meta-level thinking. It's super awkward, because you have to simultaneously comprehend your build on the Scala value level, the Scala type level, and the SBT mutable engine level. The fact that you have to learn .sbt and how it differs from a regular .scala file is an additional speed bump in this process, and I would not underestimate how intimidating that is to a beginner.

As a genuine question, what's the core problem that .sbt solves? As far as I can see, the answer is that the alternative would be a comma-separated list of project settings, with an inability to define variables and methods inline. Perhaps I'm missing something there. But if that's the case, then I think the issue is ultimately that we're trying to avoid plain old Scala value-level mutability by modeling it in an SBT meta-language. I get that the result of this is an ability to introspect the build in interesting ways, but could the same thing not be accomplished by making Setting mutable and but giving it the logic to remember its own changes?

I hope the maintainers don't take this the wrong way, because it is meant as constructive criticisms, but what I'm describing here is a microcosm of the challenges the Scala ecosystem faces. The core projects are largely written by experts, and I think sometimes it's difficult for experts to look at their own constructs through the eyes of a non-expert. There many parts of the ecosystem where this is less necessary, but I would argue that as central as SBT is, this is a place where that principle should really be applied.

@eed3si9n
Copy link
Member Author

eed3si9n commented Apr 3, 2016

@acjay Thanks for writing this.

The core projects are largely written by experts, and I think sometimes it's difficult for experts to look at their own constructs through the eyes of a non-expert. There many parts of the ecosystem where this is less necessary, but I would argue that as central as SBT is, this is a place where that principle should really be applied.

We are in agreement on this point. The recent history of sbt, especially during sbt 0.13, has been largely driven by making the shock of sbt 0.10 more palatable through auto plugins, := macros, and improving parsing (removing blank lines). Also note that sizable portion of sbt user are Java developers (mainly via Play Java). I don't have hard evidence, but anecdotally sbt 0.13 changes have significantly improved the learning curve and getting-started problems of sbt. This is not to say that there is no learning involved with sbt.

SBT is more or less mandatory to anybody starting a serious Scala project, and it's really difficult to avoid immediately being confronted by a whole new world of unfamiliar concepts and meta-level thinking.

Once someone starts writing their build definition, they need to face the fact that build DSL is not Scala, and learn it as a separate thing, like regex or CSS. Tasks are dependency graph of calculations that gets executed in parallel. The earlier we can convey that build DSL is not Scala (but it can call Scala, similar to Scala is not Java), the more we can avoid the confusion of why := is needed.

The fact that you have to learn .sbt and how it differs from a regular .scala file is an additional speed bump in this process, and I would not underestimate how intimidating that is to a beginner.

Like snowboarding, the first day is going to suck, and the second day it gets better. I don't think you have to comprehend all the inner workings of sbt on day 1, and one should be able to get started on build.sbt by copy-pasting. I also don't think changing the file extension from .sbt to .scala is going make this problem any better.

As a genuine question, what's the core problem that .sbt solves?

It's a shorter form of writing the build DSL. Take a look at the following project/assembly.sbt:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.2")

It's shorter because it automatically imports the necessary names and implicits, and auto plugin also allows the plugin authors to introduce custom names and settings into the build. This is more copy-pastable than project/project/Build.scala equivalent you'd have to write if we get rid of .sbt.

People often poke fun about "simple" part of sbt (Insert simple != easy). We have a redundant functionality called build.sbt and project/Build.scala, and we are trying to get rid of one to reduce the cognitive load.

The .sbt layer is too clever by half, and it's a shame to see the team doubling down on it.

As commented above, the removal of Build.scala is as much about Project(..., settings = ) constructor. Since this was the way to define a subproject at the time project/Build.scala came out, the overlap of project/Build.scala and Project(..., settings = ) constructor usage is high. I'm going to remove Project(..., settings = ) constructor soon, so that'll require those users to change the build file anyway.

There are plugins that depend on the other plugins. When this happens the setting sequences must be loaded in some specific order, otherwise the plugins would break. From a build user who just wants to compile CoffeeScript how the plugins are related is an incidental complexity. Auto plugins solve this by the build users naming the plugins, and sbt ordering the settings based on the plugin dependencies.

@fommil
Copy link
Contributor

fommil commented Apr 3, 2016

I don't understand how removing .scala support removes Project(..., settings = ). Surely that is still possible from a .sbt file and if you remove the ability to provide the settings in a Project constructor, doesn't that allow us to continue to use .scala?

I challenge your claim that .scala files have more use of the settings constructor. When I made an aggressive change to ensime-sbt which broke if anybody used the settings constructor, almost everybody was doing it with .sbt files.

The only way to fix rogue usages of Project(..., settings = ) is to make that constructor private.

@eed3si9n
Copy link
Member Author

eed3si9n commented Apr 6, 2016

@fommil I forgot to mention the obvious issue with sbt.Build: override lazy val settings = .... Here's a live example of the mess - xerial/sbt-sonatype#28
Neither plugin author nor the build user can be blamed for falling into this trap.

@ralph-tice
Copy link

Nobody likes to see "their" feature or code path deprecated, and I personally am a big fan and proponent of Build.scala. But strictly from a maintainability standpoint, I don't think any open source project has the bandwidth to cater to a minority of users, nor should they. So I tried to find some sort of proxy for usage metrics, and the results (below) surprised me:

I think these searches are representative in that they count roughly similar things, and the data surprised me. I probably would not be using Build.scala today if there were some usage metrics available to help me choose what the community has de-facto voted for, and for me the schism between Build.scala and build.sbt is more confusing than it would have been to learn the build DSL in the first place.

It might be worthwhile to consider a sbt survey task to allow users to voluntarily report their feature usage metrics.

@DavidPerezIngeniero
Copy link

I'm not using Build.scala anymore

@japgolly
Copy link
Contributor

For others concerned about this like I was, this is much less of a deal than I thought it was going to be.

If you haven't seen this PR for Scala.JS I suggest you take a look. You basically just turn build.sbt into an easy-to-read table of contents, rest of build stays the same.

giabao added a commit to giabao/scalac-scoverage-plugin that referenced this pull request Apr 27, 2016
+ see sbt/sbt#2524
+ use sbt-sonatype plugin
+ update scalastyle-sbt-plugin 0.8.0, sbt-pgp 1.0.0
+ update scala 2.11.8, mokito 1.10.19, scalatest 2.2.6, joda-time 2.9.3, joda-convert 1.8.1, scala-xml 1.0.5
+ remove build setting `javacOptions`. We have no .java files
+ remove Scoverage's `lazy val LocalTest = config("local") extend Test`
@eed3si9n eed3si9n modified the milestones: 1.0.0, 1.0 May 13, 2016
@eed3si9n eed3si9n mentioned this pull request Sep 6, 2016
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.