cuda: fix conflict statements for x86-64 targets by alalazo · Pull Request #13472 · spack/spack

alalazo · 2019-10-28T16:47:17Z

This build system mixin was not updated after the support for specific targets has been merged.

fixes spack#13462 This build system mixin was not updated after the support for specific targets has been merged.

alalazo · 2019-10-28T16:47:51Z

@glennpj Can you confirm this fixes your issue in #13462 ?

glennpj · 2019-10-28T17:16:28Z

@alalazo No, it does not resolve the issue. The concretizer will still succeed with gcc@9.1.0 rather than returning the conflict.

alalazo · 2019-10-28T17:31:48Z

@glennpj Here's what I tried:

$ spack spec libbeagle+cuda ^cuda@10.0.130
Input spec
--------------------------------
libbeagle+cuda
    ^cuda@10.0.130

Concretized
--------------------------------
==> Error: Conflicts in concretized spec "libbeagle@3.1.2%gcc@9.0.1+cuda cuda_arch=none arch=linux-ubuntu18.04-broadwell/fdhyu3p"

List of matching conflicts for spec:

    libbeagle@3.1.2%gcc@9.0.1+cuda cuda_arch=none arch=linux-ubuntu18.04-broadwell
        ^autoconf@2.69%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
            ^m4@1.4.18%gcc@9.0.1 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-ubuntu18.04-broadwell
                ^libsigsegv@2.12%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
            ^perl@5.30.0%gcc@9.0.1+cpanm+shared+threads arch=linux-ubuntu18.04-broadwell
                ^gdbm@1.18.1%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                    ^readline@8.0%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                        ^ncurses@6.1%gcc@9.0.1~symlinks~termlib arch=linux-ubuntu18.04-broadwell
                            ^pkgconf@1.6.3%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
        ^automake@1.16.1%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
        ^cuda@10.0.130%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
        ^libtool@2.4.6%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
        ^openjdk@11.0.2%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
        ^subversion@1.12.2%gcc@9.0.1~perl+serf arch=linux-ubuntu18.04-broadwell
            ^apr@1.7.0%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
            ^apr-util@1.6.1%gcc@9.0.1+crypto~gdbm~odbc~pgsql~sqlite arch=linux-ubuntu18.04-broadwell
                ^expat@2.2.9%gcc@9.0.1+libbsd arch=linux-ubuntu18.04-broadwell
                    ^libbsd@0.9.1%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                ^libiconv@1.16%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                ^openssl@1.1.1d%gcc@9.0.1+systemcerts arch=linux-ubuntu18.04-broadwell
                    ^zlib@1.2.11%gcc@9.0.1+optimize+pic+shared arch=linux-ubuntu18.04-broadwell
            ^lz4@1.9.2%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
            ^serf@1.3.9%gcc@9.0.1~debug patches=b6593a4dafea97d1bef13b5d57fecb1410f02452d7def51b31f76bf76a85c4ad arch=linux-ubuntu18.04-broadwell
                ^scons@3.1.1%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                    ^py-setuptools@41.4.0%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                        ^python@3.7.4%gcc@9.0.1+bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=linux-ubuntu18.04-broadwell
                            ^bzip2@1.0.8%gcc@9.0.1+shared arch=linux-ubuntu18.04-broadwell
                                ^diffutils@3.7%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                            ^gettext@0.20.1%gcc@9.0.1+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-ubuntu18.04-broadwell
                                ^libxml2@2.9.9%gcc@9.0.1~python arch=linux-ubuntu18.04-broadwell
                                    ^xz@5.2.4%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                                ^tar@1.32%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                            ^libffi@3.2.1%gcc@9.0.1 arch=linux-ubuntu18.04-broadwell
                            ^sqlite@3.30.1%gcc@9.0.1~column_metadata+fts~functions~rtree arch=linux-ubuntu18.04-broadwell
            ^utf8proc@2.4.0%gcc@9.0.1 build_type=RelWithDebInfo arch=linux-ubuntu18.04-broadwell
                ^cmake@3.15.4%gcc@9.0.1~doc+ncurses+openssl+ownlibs~qt arch=linux-ubuntu18.04-broadwell

1. "%gcc@8:" conflicts with "libbeagle+cuda ^cuda@10.0.130 arch=linux-None-x86_64:"

It succeeds on develop. Can you post in the issue your environment and ping me?

glennpj · 2019-10-28T18:19:53Z

@alalazo It is not just in an environment,

$ spack spec libbeagle+cuda
Input spec
--------------------------------
libbeagle+cuda

Concretized
--------------------------------
libbeagle@3.1.2%gcc@9.1.0+cuda cuda_arch=none arch=linux-centos7-broadwell
    ^autoconf@2.69%gcc@9.1.0 arch=linux-centos7-broadwell
        ^m4@1.4.18%gcc@9.1.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-centos7-broadwell
            ^libsigsegv@2.12%gcc@9.1.0 arch=linux-centos7-broadwell
        ^perl@5.30.0%gcc@9.1.0+cpanm+shared+threads arch=linux-centos7-broadwell
            ^gdbm@1.18.1%gcc@9.1.0 arch=linux-centos7-broadwell
                ^readline@8.0%gcc@9.1.0 arch=linux-centos7-broadwell
                    ^ncurses@6.1%gcc@9.1.0~symlinks~termlib arch=linux-centos7-broadwell
                        ^pkgconf@1.6.3%gcc@9.1.0 arch=linux-centos7-broadwell
    ^automake@1.16.1%gcc@9.1.0 arch=linux-centos7-broadwell
    ^cuda@10.1.243%gcc@9.1.0 arch=linux-centos7-broadwell
    ^libtool@2.4.6%gcc@9.1.0 arch=linux-centos7-broadwell
    ^openjdk@11.0.2%gcc@9.1.0 arch=linux-centos7-broadwell
    ^subversion@1.12.2%gcc@9.1.0~perl+serf arch=linux-centos7-broadwell
        ^apr@1.7.0%gcc@9.1.0 arch=linux-centos7-broadwell
        ^apr-util@1.6.1%gcc@9.1.0+crypto~gdbm~odbc~pgsql~sqlite arch=linux-centos7-broadwell
            ^expat@2.2.9%gcc@9.1.0+libbsd arch=linux-centos7-broadwell
                ^libbsd@0.9.1%gcc@9.1.0 arch=linux-centos7-broadwell
            ^libiconv@1.16%gcc@9.1.0 arch=linux-centos7-broadwell
            ^openssl@1.1.1d%gcc@9.1.0+systemcerts arch=linux-centos7-broadwell
                ^zlib@1.2.11%gcc@9.1.0+optimize+pic+shared arch=linux-centos7-broadwell
        ^lz4@1.9.2%gcc@9.1.0 arch=linux-centos7-broadwell
        ^serf@1.3.9%gcc@9.1.0~debug patches=b6593a4dafea97d1bef13b5d57fecb1410f02452d7def51b31f76bf76a85c4ad arch=linux-centos7-broadwell
            ^scons@3.1.1%gcc@9.1.0 arch=linux-centos7-broadwell
                ^py-setuptools@41.4.0%gcc@9.1.0 arch=linux-centos7-broadwell
                    ^python@3.7.4%gcc@9.1.0+bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=linux-centos7-broadwell
                        ^bzip2@1.0.8%gcc@9.1.0+shared arch=linux-centos7-broadwell
                            ^diffutils@3.7%gcc@9.1.0 arch=linux-centos7-broadwell
                        ^gettext@0.20.1%gcc@9.1.0+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-centos7-broadwell
                            ^libxml2@2.9.9%gcc@9.1.0~python arch=linux-centos7-broadwell
                                ^xz@5.2.4%gcc@9.1.0 arch=linux-centos7-broadwell
                            ^tar@1.32%gcc@9.1.0 arch=linux-centos7-broadwell
                        ^libffi@3.2.1%gcc@9.1.0 arch=linux-centos7-broadwell
                        ^sqlite@3.30.0%gcc@9.1.0~column_metadata+fts~functions~rtree arch=linux-centos7-broadwell
        ^utf8proc@2.4.0%gcc@9.1.0 build_type=RelWithDebInfo arch=linux-centos7-broadwell
            ^cmake@3.15.4%gcc@9.1.0~doc+ncurses+openssl+ownlibs~qt arch=linux-centos7-broadwell

My spack tree is from October 25, 2019 so it is fairly recent. I will try a fresh pull.

glennpj · 2019-10-28T18:42:33Z

@alalazo There is a second issue.

$ spack spec libbeagle+cuda

will not conflict but

$ spack spec libbeagle+cuda ^cuda@10.0.130

will conflict. So, in addition to the fix for arch_platform, the conflict will need tweaking. It is currently

conflicts('%gcc@8:', when='+cuda ^cuda@10.0.130' + arch_platform)

alalazo · 2019-10-28T18:42:43Z

I don't see an issue with what you posted. The only conflict I see is:

conflicts('%gcc@8:', when='+cuda ^cuda@10.0.130')

but your spec has cuda@10.1.243%gcc@9.1.0 arch=linux-centos7-broadwell and 10.1.243 != 10.0.130. Am I missing something in the CudaPackage?

glennpj · 2019-10-28T18:49:27Z

According to https://gist.github.com/ax3l/9489132

cuda	gcc
10.0.130	<=7
10.1.105	<=8

But nothing is defined after that. From the build failure though cuda-10.1.243 does not work with gcc-9

glennpj · 2019-10-28T18:54:23Z

If the conflict statement is changed to

conflicts('%gcc@8:', when='+cuda ^cuda@:10.2' + arch_platform)

it will do the right thing. That constraint would need to be altered at some point once there is a cuda-10.2+.

alalazo · 2019-10-28T19:23:04Z

@glennpj Modification done.

glennpj · 2019-10-28T19:41:50Z

@alalazo Is it necessary to make modifications for the power architecture as well?

arch_platform = ' arch=ppc64le platform=linux'

alalazo · 2019-10-28T19:48:00Z

Thanks for the hint @glennpj. It looks like ppc64le needed an update too. I don't see anything else that matches the pattern arch_platform = * but let me know if you think CudaPackage needs further modifications.

lib/spack/spack/build_systems/cuda.py

alalazo · 2019-10-29T09:16:54Z

@ax3l @glennpj Should be ready for another review. Waiting for your green light to merge.

lib/spack/spack/build_systems/cuda.py

Co-Authored-By: Axel Huebl <axel.huebl@plasma.ninja>

* cuda: fix conflict statements for x86-64 targets fixes #13462 This build system mixin was not updated after the support for specific targets has been merged. * Updated the version range of cuda that conflicts with gcc@8: * Updated the version range of cuda that conflicts with gcc@8: for ppc64le * Relaxed conflicts for version > 10.1 * Updated versions in conflicts Co-Authored-By: Axel Huebl <axel.huebl@plasma.ninja>

cuda: fix conflict statements for x86-64 targets

1ad1629

fixes spack#13462 This build system mixin was not updated after the support for specific targets has been merged.

alalazo added platform-support cuda conflicts labels Oct 28, 2019

alalazo requested review from adamjstewart and becker33 October 28, 2019 16:47

adamjstewart approved these changes Oct 28, 2019

View reviewed changes

Updated the version range of cuda that conflicts with gcc@8:

0a220a4

alalazo requested a review from ax3l October 28, 2019 19:23

Updated the version range of cuda that conflicts with gcc@8: for ppc64le

67911d2

ax3l reviewed Oct 28, 2019

View reviewed changes

lib/spack/spack/build_systems/cuda.py Outdated Show resolved Hide resolved

ax3l reviewed Oct 28, 2019

View reviewed changes

lib/spack/spack/build_systems/cuda.py Outdated Show resolved Hide resolved

Relaxed conflicts for version > 10.1

d88358b

ax3l reviewed Oct 29, 2019

View reviewed changes

Updated versions in conflicts

0333e7c

Co-Authored-By: Axel Huebl <axel.huebl@plasma.ninja>

alalazo self-assigned this Oct 30, 2019

ax3l approved these changes Oct 30, 2019

View reviewed changes

ax3l merged commit cf1c381 into spack:develop Oct 30, 2019

alalazo deleted the fixes/constraints_on_cuda branch October 30, 2019 16:34

Conversation

alalazo commented Oct 28, 2019

Uh oh!

alalazo commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

alalazo commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

alalazo commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

alalazo commented Oct 28, 2019

Uh oh!

glennpj commented Oct 28, 2019

Uh oh!

alalazo commented Oct 28, 2019

Uh oh!

Uh oh!

Uh oh!

alalazo commented Oct 29, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants