Skip to content

Off-the-grid installation of sbt#2564

Merged
eed3si9n merged 10 commits intosbt:0.13from
Duhemm:wip/fix-2518
Apr 20, 2016
Merged

Off-the-grid installation of sbt#2564
eed3si9n merged 10 commits intosbt:0.13from
Duhemm:wip/fix-2518

Conversation

@Duhemm
Copy link
Contributor

@Duhemm Duhemm commented Apr 18, 2016

This PR adds a new main object sbt.Main that implements a launcher that can start sbt without ivy or internet access.

The new command install creates the fat JAR that can then be launched.

Ref #2518

Duhemm added 10 commits April 4, 2016 13:35
This commit introduces a new "static" launcher that does not use Ivy to
gather all the artifacts that it requires, but rather expect them to be
immediately available.

To be able to use sbt without Internet access, we add a new
`ComponentCompiler` that is able to retrieve the bridge sources from the
resources on classpath and compile it.
The launcher defines a top classloader that willbe used by all
`ScalaInstance`s. Previously, this top classloader had a parent that
contained the scala library 2.10, which prevented the correct
compilation of the compiler bridge for scala 2.11.

Also, we no longer need the scala-reflect JAR.
It turns out we need to have `scala-reflect.jar` on classpath to compile
the compiler bridge for the static scala instance of the launcher.
It turns out that we can leverage the`FakeResolver` that has been
implemented to use with the static launcher, and resolve a "fake
compiler bridge" using it, rather than copying it from the resources.

This also has the advantage of not requiring to change the build
definition.
This allows sbt to resolve the compiler bridge sources when using the
static launcher
@eed3si9n eed3si9n merged commit b10414d into sbt:0.13 Apr 20, 2016
@eed3si9n
Copy link
Member

Notes please.
Also forward port.

Duhemm added a commit to Duhemm/sbt that referenced this pull request Apr 26, 2016
Duhemm added a commit to Duhemm/librarymanagement that referenced this pull request Apr 27, 2016
This is a combination of 13 commits. I squashed these 13 commits to make
forward porting those changes easier, since some commit undo the changes
of other commits.

The PRs that include these changes can be found at
sbt/sbt#2564 and
sbt/sbt#2576.

Static launcher, get bridge sources from resources

This commit introduces a new "static" launcher that does not use Ivy to
gather all the artifacts that it requires, but rather expect them to be
immediately available.

To be able to use sbt without Internet access, we add a new
`ComponentCompiler` that is able to retrieve the bridge sources from the
resources on classpath and compile it.

Fix classpath issues in static launcher

The launcher defines a top classloader that willbe used by all
`ScalaInstance`s. Previously, this top classloader had a parent that
contained the scala library 2.10, which prevented the correct
compilation of the compiler bridge for scala 2.11.

Also, we no longer need the scala-reflect JAR.

Tests for FakeResolver

Add `scala-reflect.jar` to JARs of `StaticScalaProvider`

It turns out we need to have `scala-reflect.jar` on classpath to compile
the compiler bridge for the static scala instance of the launcher.

Comply to Ivy's specification in `FakeResolver`

Remove `CompilerBridgeProvider` and `ResourceBridgeProvider`

It turns out that we can leverage the`FakeResolver` that has been
implemented to use with the static launcher, and resolve a "fake
compiler bridge" using it, rather than copying it from the resources.

This also has the advantage of not requiring to change the build
definition.

Fix NPE in FakeResolver

Add compiler bridge sources to fake resolver

This allows sbt to resolve the compiler bridge sources when using the
static launcher

Don't hardcode sbt version in static launcher

Add scala compiler and library to fake resolver

This allows us to still resolve them if we have no other resolver
configured.

Add `RepositoriesParser`

This parser is used by the static launcher to parse the definition of
resolvers that override the build resolvers.

Support repositories override in static launcher

The static launcher will now parse user-defined repositories like the
usual launcher does.

Specifically, the static launcher now uses the following configuration:

 - `sbt.boot.directory`: specifies the boot directory that sbt will use.
   Defaults to `~/.sbt/boot`.
 - `sbt.override.build.repos`: indicate whether we want to override the
   build resolvers. Defaults to false.
 - `sbt.repository.config`: specifies the path to the files that
   contains repositories definition. Defaults to
   `${sbt.boot.directory}/repositories`.

Notes for sbt/sbt#2564 & sbt/sbt#2576
@eed3si9n eed3si9n mentioned this pull request May 1, 2016
7 tasks
@k0001
Copy link

k0001 commented Jul 29, 2016

If I understand correctly, among other things this change allows me to call, for example, sbt compile within a SBT project (or a similar command that avoids sbt-launcher perhaps), and provided all the jars that both sbt and the project itself depends on are already in place, the project will start compiling without downloading anything. Is this correct?

If this is the case: How do I tell sbt where to find all those jars? Otherwise: Can I accomplish the example mentioned above somehow?

Sorry if this isn't the right place to ask this question!

@k0001
Copy link

k0001 commented Jul 29, 2016

FYI: I'm working on improving the packaging of SBT applications in the Nix package manager (https://nixos.org/nix). Basically, I want to be able to call sbt compile (or any other command that would achieve the same thing), without having SBT hit the internet. One valid approach seems to be to create a local ivy cache, but that's really cumbersome to do as you have to recreate the ivy repository structure and have the ivy.xml files. Instead, I would like to be able to somehow list where to find all of the jars both SBT and the project itself needs by hand, in a CLASSPATH-like fashion, and it looks like this pull request tries to solve some of that. Am I right? Thanks!

@eed3si9n
Copy link
Member

This was an experimental feature, and it hasn't been given enough attention.
I wrote https://github.com/sbt/sbt-export-repo, which lets you create a preloaded local repository correctly, by driving Ivy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants