Skip to content

[sbt 1.0] Support sourceGenerators += Def.task { ... }#2942

Merged
eed3si9n merged 2 commits intosbt:1.0.xfrom
eed3si9n:topic/generators
Feb 6, 2017
Merged

[sbt 1.0] Support sourceGenerators += Def.task { ... }#2942
eed3si9n merged 2 commits intosbt:1.0.xfrom
eed3si9n:topic/generators

Conversation

@eed3si9n
Copy link
Copy Markdown
Member

This is on top of #2937

During the docs review, one of the comments we got from @lihaoyi was:

Despite writing many sourceGenerators, it's not clear what this whole Def.task{}.taskValue thingy is about, even after reading this snippet. Why can we not directly += the result of myGenerator?

A quick answer is because the types don't match up. The expected type for the generator is Seq[Task[Seq[File]]] whereas Def.task { ... } returns Def.Initialize[Task[...]]. Initialize[_] is sort of the DAG node, and Task[_] is like a beefed up Function0[_].

Since we are pretty sure what user wants, we can extract the Task[...] out of Initialize[Task[...]].

Append instance that extracts taskValue

This adds a macro-level hack to support += op for sourceGenerators and resourceGenerators using RHS of Initialize[Task[Seq[File]]]. When the types match up, the macro now calls .taskValue automatically.

This adds a macro-level hack to support += op for sourceGenerators and resourceGenerators using RHS of Initialize[Task[Seq[File]]].
When the types match up, the macro now calls `.taskValue` automatically.
Copy link
Copy Markdown
Member

@dwijnand dwijnand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we're improving or not the situation by adding a macro-level hack.

@eed3si9n
Copy link
Copy Markdown
Member Author

At least the end-user won't have to think about the subtlety of "task" vs Def.Initialize[Task[...]] vs Task[...]. I think that's a progress.

@dwijnand
Copy link
Copy Markdown
Member

dwijnand commented Jan 24, 2017

This would be like a macro-level hack so you could do scalaVersion := Some("2.12.1") by just unwrapping the Some.

Personally I think this would be just as easily solved with a Def.taskValue, but feel free to overrule and merge.

@eed3si9n
Copy link
Copy Markdown
Member Author

My real target is the backport to 0.13.14, so we can address both of the issues that came up in 0.13.13 regarding <<= deprecation (this and #1444).

@dwijnand
Copy link
Copy Markdown
Member

dwijnand commented Jan 24, 2017

I think instead we should provide Def.taskValue { .. } as a "solution" to the criticism on Def.task { .. }.taskValue. And then add -Ywarn-value-discard by default to the meta-build scalacOptions, like we added -deprecation.

@eed3si9n
Copy link
Copy Markdown
Member Author

I don't see Def.taskValue { Nil } as an improvement over (Def.task { Nil }).taskValue since you're still introducing additional concept to migrate from <+= without apparent return to the user. Worse, this would add discrepancy with task key. The test for this PR supports:

sourceGenerators in Compile += buildInfo,
sourceGenerators in Compile += Def.task { Nil }

@dwijnand
Copy link
Copy Markdown
Member

Worse, this would add discrepancy with task key.

What discrepancy? What do you mean?

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Feb 5, 2017

Worse, this would add discrepancy with task key.

What discrepancy? What do you mean?

In general this is where the rubber meets the road, or (insert impedance mismatch analogy).
We teach people about settings vs tasks, there are keys specific to settings and tasks, but the actual types around "settings" and "tasks" are more complicated.
What we colloquially call "settings" are actually Def.Initialize[A]. Note SettingKey[A] extends KeyedInitialize[A].

Similarly, what we colloquially call "tasks" are Def.Initialize[Task[A]]. TaskKey[A] extends KeyedInitialize[Task[A]]. This is how we are able to maintain the consistent feel of custom tasks Def.task { ... } and built-in task (keys).

My claim is that the build user expects sourceGenerators to also collect reference to "tasks", and that its type is SettingKey[Seq[Task[Seq[File]]]] is not that important. In other words, I should be able to write:

sourceGenerators in Compile += buildInfo

@dwijnand
Copy link
Copy Markdown
Member

dwijnand commented Feb 5, 2017

Should you also be able to write crossScalaVersions += scalaVersion? crossScalaVersions collects version strings and scalaVersion is a version string.

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Feb 5, 2017

Should you also be able to write crossScalaVersions += scalaVersion? crossScalaVersions collects version strings and scalaVersion is a version string.

No, I don't think so. We know crossScalaVersions is SettingKey[Seq[String]] as in a "setting of sequence of String", and scalaVersion a "setting of String". So it's perfectly consistent with Getting Started narrative that it requires .value in:

crossScalaVersions += scalaVersion.value

The tricky bit is that sourceGenerators is SettingKey[Seq[Task[Seq[File]]]], a "setting of sequence of Tasks". I'm trying to smooth out the awkwardness that Task is actually not "task".

@dwijnand
Copy link
Copy Markdown
Member

dwijnand commented Feb 6, 2017

Let's be clear, sourceGenerators and resourceGenerators are outliers that expose sbt's underlying Task type. We should either (a) own this complexity and detail/document it more/better or (b) address the complexity. For instance why can't they, instead of be SettingKey[Seq[Task[Seq[File]]]], be TaskKey[Seq[File]]? I don't think we should pick option (c): obscure it, by introducing false Append providers and a macro-level implicit conversion.

If it's accepted that you can't crossScalaVersions += scalaVersion but you must crossScalaVersions += scalaVersion.value, then it's equally accepted that sourceGenerators in Compile += buildInfo doesn't make any sense, and that if you want the underlying value (Seq[File) then you buildInfo.value, and if you want the underlying task then you buildInfo.taskValue.

And if adding an inline task by writing sourceGenerators in Compile += Def.task { .. }.taskValue is too annoying let's add Def.taskValue { .. }.

Or, of course, it's your prerogative to overrule me.

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Feb 6, 2017

Or, of course, it's your prerogative to overrule me.

I'm fairly sure this magic is the way to go.

For a reference, earlier version of sbt required seq(...) function to put Seq[Setting[_]] in build.sbt. This was worked around by implicit magic in f8d12c5 that unified Seq[Setting[_]] and Setting[_] into SettingsDefinition. We still carry that over today - https://github.com/sbt/util/blob/5306e292900676f69dde458b9a2d7223e3a4c141/internal/util-collection/src/main/scala/sbt/internal/util/Settings.scala#L426-L434

In this case, the implicit conversion from "task" to Task can be considered analogous of eta expansion in Scala. This matches user expectation as we see in these comments sbt/website#306 (comment):

Despite writing many sourceGenerators, it's not clear what this whole Def.task{}.taskValue thingy is about, even after reading this snippet. Why can we not directly += the result of myGenerator?

@eed3si9n eed3si9n merged commit fc92bc5 into sbt:1.0.x Feb 6, 2017
@eed3si9n eed3si9n deleted the topic/generators branch February 6, 2017 22:36
@fommil
Copy link
Copy Markdown
Contributor

fommil commented Feb 7, 2017

why have a macro at all, isn't this ideal for an implicit conversion?

@dwijnand
Copy link
Copy Markdown
Member

dwijnand commented Feb 7, 2017

A few thoughts:

  1. Do implicit conversions kick-in post macro-expansion?
  2. This "conversion" can only happen within the context of sbt's macros, you can't generally call Initialize[Task[T]].taskValue, so I don't think you can define the implicit conversion
  3. Having it as a macro-level hack at least scopes it to only this codepath, rather than anywhere.

@eed3si9n
Copy link
Copy Markdown
Member Author

eed3si9n commented Feb 7, 2017

I am doing an implicit conversion. It's just that the RHS requires a macro, so it's stubbed with ???.

@jvican
Copy link
Copy Markdown
Member

jvican commented Mar 30, 2017

One thought: is the failure in CI relevant? I believe it is.

@eed3si9n
Copy link
Copy Markdown
Member Author

iirc I cancelled the Travis job on merge commit because there was some other thing I needed to verify at the time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants