xsdk: Very preliminary xsdk.py definition for installing xSDK packages by BarrySmith · Pull Request #2058 · spack/spack

BarrySmith · 2016-10-19T21:33:18Z

This is a replacement for pull request #2035

After rebase over latest develop branch

I had to make a new pull request because I did not change the branch name in my fork before adding code.

tgamblin · 2016-10-19T21:34:55Z

@BarrySmith: even if you rebase in your fork's develop branch, you can still force-push and it'll show up in the PR. It's pretty nice because the old commits will go away and the new ones will appear lower in the conversation thread.

BarrySmith · 2016-10-20T00:42:02Z

@tgamblin Agreed. The problem is I made a mistake of pushing when my fork was in a state with the develop branch in my fork matching the develop branch in the LLNL repository (I had move my changes to another branch). As soon as I pushed github noted the pull request branch matched the branch in the LLNL repository and so marked the pull request as complete. I couldn't find a way to reopen a complete pull request (which makes sense). So my intention was not to start a new pull request, I just did something that prematurely closed my original pull request.

davydden

to make it easier to install develop versions everywhere, you can add to you ~/.spack/packages.yaml something like

packages:
   hypre:
        version: [develop]
   petsc:
        version: [develop]
   trilinos:
       version: [develop]
       variants: +xsdkflags
   superlu-dist:
       version: [develop]

This would essentially work as extra constraints on top of whatever you install by spack install xyz.

davydden · 2016-10-20T04:44:27Z

var/spack/repos/builtin/packages/petsc/package.py

-    depends_on('superlu-dist@:4.3', when='@:3.6.4+superlu-dist+mpi')
-    depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi')
+    depends_on('hypre@develop~internal-superlu', when='@develop+hypre+mpi~complex')
+    depends_on('superlu-dist@:4.3', when='@3.6.4+superlu-dist+mpi')


i don't think you want to change :3.6.4 to 3.6.4 here.

davydden · 2016-10-20T04:45:01Z

var/spack/repos/builtin/packages/petsc/package.py

-    depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi')
+    depends_on('hypre@develop~internal-superlu', when='@develop+hypre+mpi~complex')
+    depends_on('superlu-dist@:4.3', when='@3.6.4+superlu-dist+mpi')
+    depends_on('superlu-dist@5.0.0', when='@3.7+superlu-dist+mpi')


same here, depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi') looked correct to me.

i.e. you probably don't want to modify this line at all.

davydden · 2016-10-20T04:45:55Z

var/spack/repos/builtin/packages/petsc/package.py

    depends_on('hypre~internal-superlu', when='+hypre+mpi~complex')
-    depends_on('superlu-dist@:4.3', when='@:3.6.4+superlu-dist+mpi')
-    depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi')
+    depends_on('hypre@develop~internal-superlu', when='@develop+hypre+mpi~complex')


i think we generally do not enforce develop variants for packages which are build with develop. So i would remove this completely. If you need to build petsc with hypre@develop it should be fine to do spack install petsc ^hypre@develop.

davydden · 2016-10-20T04:46:07Z

var/spack/repos/builtin/packages/petsc/package.py

+    depends_on('hypre@develop~internal-superlu', when='@develop+hypre+mpi~complex')
+    depends_on('superlu-dist@:4.3', when='@3.6.4+superlu-dist+mpi')
+    depends_on('superlu-dist@5.0.0', when='@3.7+superlu-dist+mpi')
+    depends_on('superlu-dist@develop', when='@develop+superlu-dist+mpi')


davydden · 2016-10-20T04:46:21Z

var/spack/repos/builtin/packages/petsc/package.py

+    depends_on('superlu-dist@5.0.0', when='@3.7+superlu-dist+mpi')
+    depends_on('superlu-dist@develop', when='@develop+superlu-dist+mpi')
+    depends_on('trilinos@12.6.2', when='@3.7+trilinos+mpi')
+    depends_on('trilinos@develop', when='@develop+trilinos+mpi')


davydden · 2016-10-20T04:47:14Z

var/spack/repos/builtin/packages/petsc/package.py

+    depends_on('superlu-dist@:4.3', when='@3.6.4+superlu-dist+mpi')
+    depends_on('superlu-dist@5.0.0', when='@3.7+superlu-dist+mpi')
+    depends_on('superlu-dist@develop', when='@develop+superlu-dist+mpi')
+    depends_on('trilinos@12.6.2', when='@3.7+trilinos+mpi')


does petsc use Trilinos anyhow? Why exactly 12.6.2? It should probably also be @3.7:.

also it's not used in config below. Installation page of petsc https://www.mcs.anl.gov/petsc/documentation/installation.html does not mention trilinos.

davydden · 2016-10-20T04:48:07Z

var/spack/repos/builtin/packages/trilinos/package.py

    depends_on('scalapack', when='+mumps')
    depends_on('superlu-dist@:4.3', when='@:12.6.1+superlu-dist')
    depends_on('superlu-dist', when='@12.6.2:+superlu-dist')
+    depends_on('superlu-dist@develop', when='@develop:+superlu-dist')


davydden · 2016-10-20T04:56:24Z

just to clarify on those develop: the fact that a user installs petsc@develop (i.e. from git) does not mean that he HAS TO use hypre@develop, but of course he CAN use it as long as @develop version of hypre satisfy depends_on in petsc package. So that's why i think all those extra lines on develop should go.

davydden · 2016-10-20T05:04:10Z

btw, if the point is to control develop vs release in Xsdk package, then it will already work just fine based on depends_on inside Xsdk without those extra depends_on(hypre@develop) you are suggesting for petsc. That's because Spack will create a single graph of packages which satisfies all requirements starting from the root node (presumably Xsdk). So if you say there you need develop for all linear algebra, they will be used within the whole graph.

tgamblin · 2016-10-20T05:10:41Z

@BarrySmith: I actually agree with @davydden here -- propagating the @develop version constraint to hypre within petsc limits the user in that they can't specify an arbitrary combination of versions on the command line anymore. Doing this would cause an error:

spack install petsc@develop ^hypre@2.10.1

I think the above might be useful for testing (e.g. if you know there is some bug/regression with a particular version of something).

But, if xsdk@develop already depends on petsc@develop and hypre@develop, you can be sure that there's an easy way to get the latest dev stack.

One thing I've thought about is making the package constraints soft, so a user could override them if they wanted to (maybe petsc@develop just "prefers" hypre@develop). The concretization solver isn't quite smart enough to do that at the moment though.

davydden · 2016-10-20T05:40:24Z

there's an easy way to get the latest dev stack.

one can also add those constraints in ~/.spack/packages.yaml, which I use quite a lot.

davydden · 2016-10-20T09:25:16Z

var/spack/repos/builtin/packages/superlu-dist/package.py

    url = "http://crd-legacy.lbl.gov/~xiaoye/SuperLU/superlu_dist_4.1.tar.gz"

+    version('develop', git='https://github.com/xiaoyeli/superlu_dist', tag='master')
+    version('xsdk-0.1.0', git='https://github.com/xiaoyeli/superlu_dist', commit='0b5369f')


FYI, you can't build petsc@3.7: with superlu-dist@xsdk-0.1.0 because petsc depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi') whereas xsdk-0.1.0 < 5.0.0.

Workaround is #1983 + develop.xsdk.0.1.0 or alike.

davydden

LGTM apart from a few very minor issues.

davydden · 2016-10-21T07:25:53Z

lib/spack/env/cc

 # Unset pesky environment variables that could affect build sanity.
 #
-unset LD_LIBRARY_PATH
+#unset LD_LIBRARY_PATH


i think this does not belong to this PR and requires a separate discussion. @alalazo @tgamblin @adamjstewart ping.

I suggest creating a separate PR with this change only, or at least remove it from this PR.

Sorry this was not a "real change" for the pull request, just some intermediate mucking around.

davydden · 2016-10-21T07:26:36Z

var/spack/repos/builtin/packages/petsc/package.py

    depends_on('hypre~internal-superlu', when='+hypre+mpi~complex')
    depends_on('superlu-dist@:4.3', when='@:3.6.4+superlu-dist+mpi')
-    depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi')
+    depends_on('superlu-dist@5.0.0:', when='@3.7.0:+superlu-dist+mpi')


there is no need to write @3.7.0: the original code @3.7: will work just fine.

davydden · 2016-10-21T07:27:22Z

var/spack/repos/builtin/packages/petsc/package.py

    depends_on('superlu-dist@:4.3', when='@:3.6.4+superlu-dist+mpi')
-    depends_on('superlu-dist@5.0.0:', when='@3.7:+superlu-dist+mpi')
+    depends_on('superlu-dist@5.0.0:', when='@3.7.0:+superlu-dist+mpi')
+    depends_on('trilinos@12.6.2:', when='@3.7.0:+trilinos+mpi')


out of curiosity, how does petsc@3.7.0: use trilinos?

BarrySmith mentioned this pull request Oct 19, 2016

xsdk: Very preliminary xsdk.py definition for installing xSDK packages #2035

Closed

davydden suggested changes Oct 20, 2016

View reviewed changes

davydden reviewed Oct 20, 2016

View reviewed changes

davydden suggested changes Oct 21, 2016

View reviewed changes

citibeth changed the title ~~Replacement for pull request 2035~~ xsdk: Very preliminary xsdk.py definition for installing xSDK packages Oct 21, 2016

tgamblin added the xSDK label Oct 26, 2016

BarrySmith closed this Oct 26, 2016

Conversation

BarrySmith commented Oct 19, 2016

Uh oh!

tgamblin commented Oct 19, 2016

Uh oh!

BarrySmith commented Oct 20, 2016

Uh oh!

davydden left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davydden commented Oct 20, 2016

Uh oh!

davydden commented Oct 20, 2016

Uh oh!

tgamblin commented Oct 20, 2016

Uh oh!

davydden commented Oct 20, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davydden left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

davydden left a comment •

edited

Loading

davydden commented Oct 20, 2016 •

edited

Loading