Skip to content

[2.12] Upgrade scala-xml to latest 2.0.1#9743

Closed
mkurz wants to merge 2 commits intoscala:2.12.xfrom
mkurz:scala-xml-2.0.1
Closed

[2.12] Upgrade scala-xml to latest 2.0.1#9743
mkurz wants to merge 2 commits intoscala:2.12.xfrom
mkurz:scala-xml-2.0.1

Conversation

@mkurz
Copy link
Contributor

@mkurz mkurz commented Aug 30, 2021

@SethTisue WDYT? Maybe for the 2.12.15 release?

@scala-jenkins scala-jenkins added this to the 2.12.15 milestone Aug 30, 2021
@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

@SethTisue here you write:

scala-compiler makes only very light use of scala-xml, and scala-xml 2.0 is mostly (though not entirely) binary compatible with 1.x

@SethTisue
Copy link
Member

It's a judgment call, but I think it's better not to rock the 2.12 boat. At some point the value of consistency and stability exceeds the value of bringing something like this up to date.

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

@SethTisue I understand. However, IMHO upgrading to 2.0.1 in Scala 2.12 would make sense:
Now that people start to cross build for Scala 3, they will upgrade to scala-xml 2.0.x in their code bases. That's what we did e.g. in twirl:

val ScalaXmlVersion = "2.0.1"
ThisBuild / libraryDependencySchemes += "org.scala-lang.modules" %% "scala-xml" % VersionScheme.Always

The setting ThisBuild / libraryDependencySchemes... was necessary because, as you know, otherwise we get:

[error] (compiler / update) found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[error] 
[error] 	* org.scala-lang.modules:scala-xml_2.12:2.0.1 (early-semver) is selected over 1.0.6
[error] 	    +- com.typesafe.play:twirl-api_2.12:1.5.0+263-c57d350b+20210830-1620-SNAPSHOT (depends on 2.0.1)
[error] 	    +- org.scala-lang:scala-compiler:2.12.14              (depends on 1.0.6)

Now, on my machine, locally, I just published twirl (and play-ws, interplay) from its current master branch and tried to use that local snapshot(s) in Play's master branch. What happens now is, in Play, I can not even get into the sbt console:

$ sbt
[info] welcome to sbt 1.5.5 (Eclipse Foundation Java 11.0.12)
[info] loading global plugins from /home/mkurz/.sbt/1.0/plugins
[info] loading settings for project play2-git-build-build from buildinfo.sbt ...
[info] loading project definition from /home/mkurz/play/framework/play2-git/project/project
[info] loading settings for project play2-git-build from plugins.sbt ...
[info] loading project definition from /home/mkurz/play/framework/play2-git/project
[error] java.lang.RuntimeException: found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[error] 
[error] 	* org.scala-lang.modules:scala-xml_2.12:2.0.1 (early-semver) is selected over {1.0.6, 1.0.6}
[error] 	    +- com.typesafe.play:twirl-api_2.12:1.5.0+263-c57d350b+20210830-1634-SNAPSHOT (depends on 2.0.1)
[error] 	    +- org.scala-lang:scala-compiler:2.12.14              (depends on 1.0.6)
[error] 
[error] 
[error] this can be overridden using libraryDependencySchemes or evictionErrorLevel
[error] 	at scala.sys.package$.error(package.scala:30)
[error] 	at sbt.internal.LibraryManagement$.resolve$1(LibraryManagement.scala:89)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$12(LibraryManagement.scala:133)
[error] 	at sbt.util.Tracked$.$anonfun$lastOutput$1(Tracked.scala:73)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$20(LibraryManagement.scala:146)
[error] 	at scala.util.control.Exception$Catch.apply(Exception.scala:228)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11(LibraryManagement.scala:146)
[error] 	at sbt.internal.LibraryManagement$.$anonfun$cachedUpdate$11$adapted(LibraryManagement.scala:127)
[error] 	at sbt.util.Tracked$.$anonfun$inputChangedW$1(Tracked.scala:219)
[error] 	at sbt.internal.LibraryManagement$.cachedUpdate(LibraryManagement.scala:160)
[error] 	at sbt.Classpaths$.$anonfun$updateTask0$1(Defaults.scala:3678)
[error] 	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] 	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] 	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] 	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] 	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] 	at sbt.Execute.work(Execute.scala:291)
[error] 	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] 	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] 	at java.base/java.lang.Thread.run(Thread.java:829)
[error] (update) found version conflict(s) in library dependencies; some are suspected to be binary incompatible:
[error] 
[error] 	* org.scala-lang.modules:scala-xml_2.12:2.0.1 (early-semver) is selected over {1.0.6, 1.0.6}
[error] 	    +- com.typesafe.play:twirl-api_2.12:1.5.0+263-c57d350b+20210830-1634-SNAPSHOT (depends on 2.0.1)
[error] 	    +- org.scala-lang:scala-compiler:2.12.14              (depends on 1.0.6)
[error] 
[error] 
[error] this can be overridden using libraryDependencySchemes or evictionErrorLevel
[warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r)

To resolve that problem, I have to put

ThisBuild / libraryDependencySchemes += "org.scala-lang.modules" %% "scala-xml" % VersionScheme.Always

into project/plugins.sbt (not build.sbt(!)).

That's why I thought it might be a good idea to just upgrade scala-xml to latest 2.0.1 in the scala compiler, because I guess quite a lot of libraries will run into problems when they upgrade their scala-xml version to 2.0.1...
E.g. here in specs2 reverted back to 1.3.0 and I am pretty sure that was done because of the incompatibility with the scala 2.12 compiler... (I asked the author here, still waiting for an answer).

So that's why I think it might be a good idea to upgrade scala-xml in 2.12, to make transition to Scala 3 easier.

However maybe I am missing something here.

case e: Elem => {
val child = e.child.map(stripGroup _)
Elem(e.prefix, e.label, e.attributes, e.scope, child : _*)
Elem(e.prefix, e.label, e.attributes, e.scope, true, child : _*)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An apply method was removed here: scala/scala-xml@e73ebfd#diff-077695e8c628bc073ac8cf0a66e09dc84a283366ea94dfe11d2c4b8efef66a13L31 (which passed child.isEmpty)

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

Green 👍

@SethTisue SethTisue added the release-notes worth highlighting in next release notes label Aug 30, 2021
@SethTisue
Copy link
Member

that does seem convincing. @lrytz wdyt?

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

@SethTisue @lrytz Please also see #9744.
To avoid any problems and confusions I am totally for upgrading these two libraries in scala 2.12.15. I think that will avoid a lot of future confusions and problems.

@SethTisue
Copy link
Member

@eed3si9n any opinion on this (and the other one)? do we need to be thinking about sbt's own dependencies at all, here? any concerns about the possible effect on the sbt plugin ecosystem?

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

@SethTisue @eed3si9n Let's say we merge my two PRs here, what I would do is to upgrade sbt's Scala version to 2.12.15 afterwards (like I did here with 2.12.14: sbt/sbt#6522). When doing that, we also have to bump sbt's scala-xml version to 2.0.1:
https://github.com/sbt/sbt/blob/a199875e1dcb16cd92a439b47379954daff1a995/launcher-package/build.sbt#L37-L38

// use the scala-xml version used by the compiler not the latest: https://github.com/scala/scala/blob/v2.12.14/versions.properties#L21
lazy val scala212Xml = "org.scala-lang.modules" % "scala-xml_2.12" % "1.0.6"

If sbt is on Scala 2.12.15, we could then cut sbt 1.5.6.

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

Oh yes and https://github.com/sbt/sbt/blob/a199875e1dcb16cd92a439b47379954daff1a995/project/Dependencies.scala#L103-L104:

  val scalaXml = "org.scala-lang.modules" %% "scala-xml" % "1.3.0"
  val scalaParsers = "org.scala-lang.modules" %% "scala-parser-combinators" % "1.1.2"

@eed3si9n
Copy link
Member

If sbt is on Scala 2.12.15, we could then cut sbt 1.5.6.

I'd be against breaking (transitive) binary compatibility in a patch release. I'd be open to do that in sbt 1.6.0, but still we'd be taking some risk of compiler plugins and/or sbt plugins breaking if they used any of the dropped features of scala-xml.

If we want to make it easier to cross build, why not back publish scala-xml 1.x series to Scala 3?

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

Actually this is not really about Scala 3 (maybe I should not have mentioned that above, it's just that we are currently working on that).

This is more about sbt (1.5.x) version conflict handling. The problem occurs because the scala-compiler depends on scala-xml 1.x but if an sbt-plugin bumps its libraryDependency scala-xml version to 2.x, then you will see that error I posted above. That happens with Scala 2.12 (because plugins are published with Scala 2.12), so Scala 3 does not need to be involved.

@eed3si9n
Copy link
Member

scala-xml 1.x and 2.0.1 are incompatible according to Semantic Versioning. Maybe the actual changes that were dropped are seldom used and, in effect we can keep using it as if it is scala-xml 1.x. sbt's eviction error doesn't read between those nuanced lines, it would just look at the version numbers and try to enforce Semantic Versioning based on the opt-in intent expressed in the POM file of scala-xml.

<properties>
  <info.versionScheme>early-semver</info.versionScheme>
</properties>

This is more about sbt (1.5.x) version conflict handling.

Given that scala-compiler depends on scala-xml, and sbt depends on scala-compiler for build.sbt parsing and compiling, as far as I can tell everything is functioning as intended.

@mkurz
Copy link
Contributor Author

mkurz commented Aug 30, 2021

@SethTisue Thinking more about this problem here, I think we might made a mistake upgrading to scala-xml 2.x and scala-parser-combinators 2.x in sbt-plugins like twirl. Upgrading them to 2.x is not necessary for Scala 3 support, since sbt-plugins are build with Scala 2.12. We should just upgrade to scala-xml 2.x and scala-parser-combinators 2.x when they are actually used as libraries, because we want to cross build against Scala 3 at some point and only the 2.x versions of these libs are published for Scala 3 already.

I think what we need to do to avoid any conflicts is to downgrade to scala-xml 1.x and scala-parser-combinators 1.x if the project is a sbt-plugin.
WDYT?

@mkurz
Copy link
Contributor Author

mkurz commented Aug 31, 2021

Closing. IMHO the solution is to not use 2.x versions in sbt-plugins (because they get published for Scala 2.12 anyway)

@SethTisue
Copy link
Member

We're freshly considering this now over at scala/bug#12632.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants