Skip to content

[llvm] Update to version 10.0.0 and add new features#10295

Merged
strega-nil merged 24 commits intomicrosoft:masterfrom
yurybura:llvm10
May 4, 2020
Merged

[llvm] Update to version 10.0.0 and add new features#10295
strega-nil merged 24 commits intomicrosoft:masterfrom
yurybura:llvm10

Conversation

@yurybura
Copy link
Copy Markdown
Contributor

@yurybura yurybura commented Mar 3, 2020

LLVM 10 release is coming soon.
So, this PR updates LLVM to 10.0.0 version.
Other changes:

  • use Github monorepo;
  • support sub-projects compiler-rt, clang-tools-extra, lld, openmp, polly;
  • exclude tests and examples from build to reduce disk space usage;
  • enable RTTI by default

Related to #8521

Fixes #3377
Fixes #4277
Fixes #4461
Fixes #5153
Fixes #8347
Fixes #8463
Fixes #9670

@yurybura
Copy link
Copy Markdown
Contributor Author

yurybura commented Mar 9, 2020

Dear @PhoebeHui, could you help to solve Windows build-bot issue?
Linker generates out of memory errors:

LINK : fatal error LNK1201: error writing to program database 'C:\vsts_work\4\s\buildtrees\llvm\x64-windows-dbg\bin\llvm-dwp.pdb'; check for insufficient disk space, invalid path, or insufficient privilege

LINK : fatal error LNK1102: out of memory

LLVM is really big project. I can disable some features by default, but it isn't right way I think.

Thank you!

@PhoebeHui
Copy link
Copy Markdown
Contributor

@strega-nil, could you help take a look at above issue?

@PhoebeHui
Copy link
Copy Markdown
Contributor

PhoebeHui commented Mar 10, 2020

@yurybura, thanks for your contribution!

Noted that the following triplets disabled on https://github.com/microsoft/vcpkg/blob/master/scripts/ci.baseline.txt#L1002, if you fix the issue with specific triplet, please remove it.

llvm:arm64-windows=fail
llvm:arm-uwp=fail
llvm:x64-uwp=fail
llvm:x64-linux=ignore
llvm:x64-osx=ignore

@yurybura
Copy link
Copy Markdown
Contributor Author

@PhoebeHui, let me try to fix failed CI checks.
Thank you!

@yurybura
Copy link
Copy Markdown
Contributor Author

@PhoebeHui, I've already tested my changes for x64-windows and x64-linux triplets on my local machine.

Unfortunately I cannot test with CI bot...
After enable x64-linux CI, the build fails with the same problem (low amount of memory for link):

collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated. 

@PhoebeHui
Copy link
Copy Markdown
Contributor

llvm is a complicated and big project, it brings a stress to CI test machines, @strega-nil, any suggestions about the issue?

@JackBoosY
Copy link
Copy Markdown
Contributor

This PR is awesome, thanks for this PR!

@strega-nil
Copy link
Copy Markdown
Contributor

@PhoebeHui huh! Well, that's unfortunate. Lemme think about this!

@yurybura yurybura marked this pull request as ready for review March 12, 2020 05:46
@JackBoosY
Copy link
Copy Markdown
Contributor

Note the other paths in the cmake file.
Related: #9622.

[llvm] disable assertions and ABI breaking checks by default
[llvm] limit the maximum number of concurrent link jobs to 1
[llvm] add postfix `d` and debug shared libraries in the `/bin` directory
[llvm] don't remove debug version of tools and shared libs
@ghost ghost mentioned this pull request May 1, 2020
@alexreinking
Copy link
Copy Markdown
Contributor

@yurybura -- Thank you so much for your work on this! I'm with the Halide project and (among other things) am working on improving our build on Windows. This is a huge help.

@strega-nil
Copy link
Copy Markdown
Contributor

This is extremely exciting! I'm hopeful we can merge today :3

@tjysdsg
Copy link
Copy Markdown

tjysdsg commented May 4, 2020

Is this PR ready to be merged? I have a project that needs LLVM-10, and it would be great to vcpkg to handle this dependency! Thanks for all your hard work btw

@alexreinking
Copy link
Copy Markdown
Contributor

Is this PR ready to be merged? I have a project that needs LLVM-10, and it would be great to vcpkg to handle this dependency! Thanks for all your hard work btw

Ditto! Happy to lend a hand any way I can to grease the skids here, too 🙂

@strega-nil
Copy link
Copy Markdown
Contributor

Ah, sorry! I got distracted and forgot to hit the button... unfortunately I'm still learning the ropes -.-

@strega-nil strega-nil merged commit cfc0599 into microsoft:master May 4, 2020
@alexreinking
Copy link
Copy Markdown
Contributor

Ah, sorry! I got distracted and forgot to hit the button... unfortunately I'm still learning the ropes -.-

No, no, thank you so much! This is a big help to me and my team!

Jimmy-Hu added a commit to Jimmy-Hu/vcpkg that referenced this pull request May 4, 2020
[llvm] Update to version 10.0.0 and add new features (microsoft#10295)
@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 4, 2020

sadly, llvm is broken on Linux now.

@strega-nil
Copy link
Copy Markdown
Contributor

@cenit looks like running out of space, so I'm gonna rerun it. This is the issue with LLVM

@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 4, 2020

It’s broken also on my local machine now. Still not investigated the reason, but for sure not disk space

@strega-nil
Copy link
Copy Markdown
Contributor

That's really odd... I'm looking into it

@yurybura
Copy link
Copy Markdown
Contributor Author

yurybura commented May 4, 2020

My local build works fine. Debug build log is attached:
install-x64-linux-dbg-out.log

@cenit Could you attach your log?

@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 4, 2020

Sorry I lost logs in the update (last step is cleanup of buildtrees folder in any case). In case the same problem happens again, tonight, I will let you know. Seeing that it was broken also on master made me maybe too quick to think it was this PR fault. Maybe it was just a double coincidence (two different unrelated problems here and on my machine)

@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 4, 2020

re-launched manually to have logs sooner. Build fails very soon, here it is the config-x64-linux-dbg-err.log (CMake was still doing initial configuration)

/usr/bin/ar: creating t.a
CMake Error at lib/WindowsManifest/CMakeLists.txt:11 (get_filename_component):
  get_filename_component unknown component optimized


-- Version: 0.0.0
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success

Distributor ID: Ubuntu
Description: Ubuntu 20.04 LTS
Release: 20.04
Codename: focal

@strega-nil
Copy link
Copy Markdown
Contributor

strega-nil commented May 5, 2020

It's working on my machine... how do you run the build, and what are your environment variables?

@BillyONeal
Copy link
Copy Markdown
Member

Under the hypothesis that it failed in CI due to the OOM killer, this makes the test VMs have more RAM: #11174 (The PR infrastructure under which this PR passed had half as much RAM per core)

@strega-nil
Copy link
Copy Markdown
Contributor

The LLVM in the new infra is now failing in the same way as in @cenit's case. This is really, really weird.

@BillyONeal
Copy link
Copy Markdown
Member

Considering it turns out not to be the OOM killer and we have direct customer feedback of it being broken on their machine, should we back this out?

@strega-nil
Copy link
Copy Markdown
Contributor

Yeah, I guess so. I don't like having such an old version, but we can at least try and figure out why it's broken.

@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 5, 2020

For me it's ok to just wait a little bit and have llvm 10 properly installed. It just depends on timing, maybe @yurybura can help?

@yurybura
Copy link
Copy Markdown
Contributor Author

yurybura commented May 5, 2020

@cenit Your issue is related to the external dependency to the libxml2. It easy to fix, but I need full CMake log. This is CMake configuration issue.
@BillyONeal @strega-nil CI linker fail is not related to the issue mentioned above.

@yurybura
Copy link
Copy Markdown
Contributor Author

yurybura commented May 5, 2020

@cenit @BillyONeal @strega-nil I found the full log in the PR #11174. Let me try to fix it.

@cenit
Copy link
Copy Markdown
Contributor

cenit commented May 5, 2020

thank you. I'd really like to help you but I am really very constrained with time... sorry

@yurybura
Copy link
Copy Markdown
Contributor Author

yurybura commented May 5, 2020

@cenit @BillyONeal @strega-nil Please review/merge my fix in the PR #11180.
Thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

info:needs-maintainer-attention Lets the current 'on rotation' vcpkg maintainer know they need to look at this.

Projects

None yet