Skip to content

mesa: re-written to new meson build system #10482

Merged
opadron merged 13 commits intospack:developfrom
chuckatkins:new-mesa
May 1, 2019
Merged

mesa: re-written to new meson build system #10482
opadron merged 13 commits intospack:developfrom
chuckatkins:new-mesa

Conversation

@chuckatkins
Copy link
Copy Markdown

@chuckatkins chuckatkins commented Jan 30, 2019

  • Re-write mesa package to use the new meson build system
  • Drop backward compatibility with old mesa versions since meson wasn't reliably usable for the spack use cases of mesa until this release.
  • Express the swr driver as a variant
  • Additionally add the glx and glu virtual packages since they can be provided by a vendor's binary driver.
  • Drop the hardware drivers for now (they can be added later, but I don't believe there's much of a use case at the moment)
  • Rework downstream dependees to accommodate the changes.

Also supersedes #11301

Copy link
Copy Markdown
Member

@opadron opadron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the style checks come back green.

@chuckatkins chuckatkins changed the title WIP: mesa: re-writen to new meson build system mesa: re-written to new meson build system Apr 30, 2019
@chuckatkins
Copy link
Copy Markdown
Author

I believe we've got all the pieces to this but since it's somewhat disruptive then I'll be sure that myself and @opadron are responsive to any downstream issues that crop up because of this over the next few weeks.

@opadron
Copy link
Copy Markdown
Member

opadron commented Apr 30, 2019

@chuckatkins Looks like all virtual packages are checked for a default provider, and egl does not have one? Should it?

@chuckatkins
Copy link
Copy Markdown
Author

Let's just go ahead and drop the egl virtual package entirely for now since it's really more involved and out of scope. We'll need a separate pr to start dealing with the associated infrastructure and how that plays with glvnd.

@chuckatkins chuckatkins mentioned this pull request Apr 30, 2019
@opadron
Copy link
Copy Markdown
Member

opadron commented Apr 30, 2019

@alalazo @adamjstewart how strict are we about the failing codecov check? I'm not sure what to do about it since the codecov page with the details does not load for me.

This is ready for merge, otherwise.

Copy link
Copy Markdown
Member

@ax3l ax3l left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, sounds good to me. Did you try the affected packages like geant4 still build and work?

@ax3l
Copy link
Copy Markdown
Member

ax3l commented May 1, 2019

cc @briedel in case you want to double-check the geant4 changes

@opadron
Copy link
Copy Markdown
Member

opadron commented May 1, 2019

I went ahead and tried installing geant4 %gcc@7 +opengl cxxstd=11. I'm not sure how I'd go about testing the build, but I can say that everything built and installed without any issues.

@opadron opadron merged commit 9f84820 into spack:develop May 1, 2019
@opadron
Copy link
Copy Markdown
Member

opadron commented May 1, 2019

Since the changes are approved, and I was able to build geant4 without any issues. I'm going ahead with the merge. We'll keep an eye out for any issues with downstream packages.

@odoublewen
Copy link
Copy Markdown
Contributor

Hello. I have a nightly build of spack packages that my company uses, and last night it broke on the mesa package. I don't have a lot of time to dig into this right now, but I thought I'd at least report it.

==> Installing mesa
==> Searching for binary cache of mesa
==> Finding buildcaches in /bifx/apps/spack/mirror/build_cache
==> No binary for mesa found: installing from source
==> Cloning git repository: https://gitlab.freedesktop.org/mesa/mesa.git at tag mesa-19.0.0
==> No checksum needed when fetching with git
==> Already staged mesa-19.0.0-xdal4fbjqmrinv36x56uu4k7d6rmejwj in /tmp/bioinfo-cimr-worker-dir/workspace/bfx_spacktree_develop/spack/var/spack/stage/mesa-19.0.0-xdal4fbjqmrinv36x56uu4k7d6rmejwj
==> No patches needed for mesa
==> Building mesa [MesonPackage]
==> Executing phase: 'meson'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 1:
    'ninja' '-j36' '-v'

The build log is 1000+ lines long, and I don't immediately see a smoking gun. This is using gcc-8.2.0 & ninja-1.9.0

Please let me know how I can help.

@chuckatkins
Copy link
Copy Markdown
Author

@odoublewen Thanks for the report. What platform is this on (centos7, sles12, etc.)? Are you building the default spec or are you using any particular variants or constraints?

@chuckatkins chuckatkins deleted the new-mesa branch May 2, 2019 14:46
@opadron
Copy link
Copy Markdown
Member

opadron commented May 2, 2019

Just to offer another data point, I tried installing

mesa @19.0.0 %gcc@8 ^ninja@1.9.0

on a centos7 system, and didn't run into any build problems.

@odoublewen
Copy link
Copy Markdown
Contributor

It is centos7, as I should have mentioned.

And yes, we do specify both variants and constraints. The intersection of our packages.yaml file and the output of spack spec mesa are these three items (as far as I can tell by eyeballing it):

  pcre:
    variants: +jit
  perl:
    version: [5.26.2]
  python:
    version: [3.6.5]

Here's the full output of spack spec mesa:

Input spec
--------------------------------
mesa

Concretized
--------------------------------
mesa@19.0.0%gcc@8.2.0 buildtype=release +glx+llvm+opengl~opengles+osmesa swr=none arch=linux-centos7-x86_64
    ^binutils@2.31.1%gcc@8.2.0+gold~headers~libiberty+nls~plugins arch=linux-centos7-x86_64
        ^gettext@0.19.8.1%gcc@8.2.0+bzip2+curses+git~libunistring+libxml2 patches=9acdb4e73f67c241b5ef32505c9ddf7cf6884ca8ea661692f21dca28483b04b8 +tar+xz arch=linux-centos7-x86_64
            ^bzip2@1.0.6%gcc@8.2.0+shared arch=linux-centos7-x86_64
                ^diffutils@3.7%gcc@8.2.0 arch=linux-centos7-x86_64
            ^libxml2@2.9.8%gcc@8.2.0~python arch=linux-centos7-x86_64
                ^libiconv@1.15%gcc@8.2.0 arch=linux-centos7-x86_64
                ^pkgconf@1.6.0%gcc@8.2.0 arch=linux-centos7-x86_64
                ^xz@5.2.4%gcc@8.2.0 arch=linux-centos7-x86_64
                ^zlib@1.2.11%gcc@8.2.0+optimize+pic+shared arch=linux-centos7-x86_64
            ^ncurses@6.1%gcc@8.2.0~symlinks~termlib arch=linux-centos7-x86_64
            ^tar@1.31%gcc@8.2.0 arch=linux-centos7-x86_64
    ^expat@2.2.5%gcc@8.2.0+libbsd arch=linux-centos7-x86_64
        ^libbsd@0.9.1%gcc@8.2.0 arch=linux-centos7-x86_64
    ^libx11@1.6.7%gcc@8.2.0 arch=linux-centos7-x86_64
        ^inputproto@2.3.2%gcc@8.2.0 arch=linux-centos7-x86_64
            ^util-macros@1.19.1%gcc@8.2.0 arch=linux-centos7-x86_64
        ^kbproto@1.0.7%gcc@8.2.0 arch=linux-centos7-x86_64
        ^libxcb@1.13%gcc@8.2.0 arch=linux-centos7-x86_64
            ^libpthread-stubs@0.4%gcc@8.2.0 arch=linux-centos7-x86_64
            ^libxau@1.0.8%gcc@8.2.0 arch=linux-centos7-x86_64
                ^xproto@7.0.31%gcc@8.2.0 arch=linux-centos7-x86_64
            ^libxdmcp@1.1.2%gcc@8.2.0 arch=linux-centos7-x86_64
            ^xcb-proto@1.13%gcc@8.2.0 arch=linux-centos7-x86_64
        ^perl@5.26.2%gcc@8.2.0+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-centos7-x86_64
            ^gdbm@1.18.1%gcc@8.2.0 arch=linux-centos7-x86_64
                ^readline@7.0%gcc@8.2.0 arch=linux-centos7-x86_64
        ^xextproto@7.3.0%gcc@8.2.0 arch=linux-centos7-x86_64
        ^xtrans@1.3.5%gcc@8.2.0 arch=linux-centos7-x86_64
    ^libxext@1.3.3%gcc@8.2.0 arch=linux-centos7-x86_64
    ^llvm@8.0.0%gcc@8.2.0+all_targets build_type=Release +clang+compiler-rt+gold+internal_unwind+libcxx~link_dylib+lld+lldb+polly~python~shared_libs arch=linux-centos7-x86_64
        ^cmake@3.14.2%gcc@8.2.0~doc+ncurses+openssl~ownlibs~qt arch=linux-centos7-x86_64
            ^curl@7.63.0%gcc@8.2.0~darwinssl~libssh~libssh2~nghttp2 arch=linux-centos7-x86_64
                ^openssl@1.1.1b%gcc@8.2.0+systemcerts arch=linux-centos7-x86_64
            ^libarchive@3.3.2%gcc@8.2.0 arch=linux-centos7-x86_64
                ^lz4@1.9.0%gcc@8.2.0 arch=linux-centos7-x86_64
                ^lzo@2.10%gcc@8.2.0 arch=linux-centos7-x86_64
                ^nettle@3.4%gcc@8.2.0 arch=linux-centos7-x86_64
                    ^gmp@6.1.2%gcc@8.2.0 arch=linux-centos7-x86_64
                        ^autoconf@2.69%gcc@8.2.0 arch=linux-centos7-x86_64
                            ^m4@1.4.18%gcc@8.2.0 patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-centos7-x86_64
                                ^libsigsegv@2.11%gcc@8.2.0 arch=linux-centos7-x86_64
                        ^automake@1.16.1%gcc@8.2.0 arch=linux-centos7-x86_64
                        ^libtool@2.4.6%gcc@8.2.0 arch=linux-centos7-x86_64
            ^libuv@1.25.0%gcc@8.2.0 arch=linux-centos7-x86_64
            ^rhash@1.3.5%gcc@8.2.0 arch=linux-centos7-x86_64
        ^libedit@3.1-20170329%gcc@8.2.0 arch=linux-centos7-x86_64
        ^perl-data-dumper@2.173%gcc@8.2.0 arch=linux-centos7-x86_64
        ^py-lit@0.7.1%gcc@8.2.0 arch=linux-centos7-x86_64
            ^py-setuptools@40.8.0%gcc@8.2.0 arch=linux-centos7-x86_64
                ^python@3.6.5%gcc@8.2.0+bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter~ucs4~uuid+zlib arch=linux-centos7-x86_64
                    ^libffi@3.2.1%gcc@8.2.0 arch=linux-centos7-x86_64
                    ^sqlite@3.26.0%gcc@8.2.0~functions arch=linux-centos7-x86_64
        ^swig@3.0.12%gcc@8.2.0 arch=linux-centos7-x86_64
            ^pcre@8.42%gcc@8.2.0+jit+multibyte+utf arch=linux-centos7-x86_64
    ^meson@0.49.1%gcc@8.2.0+ninjabuild patches=d5b992925e9a927cc57d33b9ed7e61c80df9a9131d45888fbea19adcd4134e12 arch=linux-centos7-x86_64
        ^ninja@1.9.0%gcc@8.2.0 arch=linux-centos7-x86_64
    ^py-mako@1.0.4%gcc@8.2.0 arch=linux-centos7-x86_64
        ^py-markupsafe@1.0%gcc@8.2.0 arch=linux-centos7-x86_64

@odoublewen
Copy link
Copy Markdown
Contributor

I'm checking now to see if I can recreate the error with a minimal spec file.... Will update later today.

@odoublewen
Copy link
Copy Markdown
Contributor

I confirm that I get the same build error after starting from a new spack clone with vanilla configuration, running these commands:

spack install gcc@8.2.0
spack compiler add --scope site $(spack location -i gcc@8.2.0)
spack install mesa @19.0.0 %gcc@8.2.0 ^ninja@1.9.0 ^pcre+jit ^perl@5.26.2 ^python@3.6.5

@odoublewen
Copy link
Copy Markdown
Contributor

OK, I think found the (at least proximal) cause:

  >> 1043    ../src/gallium/state_trackers/glx/xlib/glx_api.c:39:10: fatal error: GL/glxproto.h: No such file or directory                                                                                           
     1044     #include <GL/glxproto.h>                                                                                                                                                                               
     1045              ^~~~~~~~~~~~~~~                                                                                                                                                                               
     1046    compilation terminated.   

hartzell pushed a commit to hartzell/spack that referenced this pull request May 2, 2019
The mesa package refers to `GL/glproto.h`.  On systems that don't have
the OS packages installed, this leads to failures during the build
[e.g. this comment in
01482](spack#10482 (comment)).

This fixes it.  Tested on a minimally provisioned CentOS 7.
@hartzell hartzell mentioned this pull request May 2, 2019
1 task
@ax3l
Copy link
Copy Markdown
Member

ax3l commented May 3, 2019

@opadron @chuckatkins feel free to see @hartzell 's proposed fix in #11360 :)

chuckatkins pushed a commit that referenced this pull request May 6, 2019
* Mesa should depend_on('glproto')

The mesa package refers to `GL/glproto.h`.  On systems that don't have
the OS packages installed, this leads to failures during the build
[e.g. this comment in
01482](#10482 (comment)).

This fixes it.  Tested on a minimally provisioned CentOS 7.

* Constrain glproto prereq to when +glx

* mesa: make glproto a build only dep
@svenevs svenevs mentioned this pull request Mar 9, 2020
2 tasks
v-dobrev added a commit that referenced this pull request Mar 27, 2020
alalazo pushed a commit that referenced this pull request Mar 28, 2020
They were originally added in #8904 and later removed (why?) in #10482.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants