Compile LLVM with Clang on release builders by alexcrichton · Pull Request #50200 · rust-lang/rust

alexcrichton · 2018-04-24T17:07:14Z

Attempting to cache in on some rustc compile time wins mentioned in #49879 (comment)

WIP: Compile LLVM with Clang on release builders Attempting to cache in on some rustc compile time wins mentioned in #49879 (comment)

alexcrichton · 2018-04-24T17:08:00Z

Gonna run this through perf to see if we can verify the build improvements. Unfortunately I wasn't seeing much of an improvement locally, so I'm hoping that a more controlled environment like perf.r-l.o can show some better numbers perhaps.

bors · 2018-04-25T14:42:35Z

💔 Test failed - status-travis

kennytm · 2018-05-09T17:03:48Z

Same error, maybe legit?

alexcrichton · 2018-05-09T20:35:49Z

Indeed yeah, it's a legit issue.

I can reproduce the hang locally, and hilariously enough the hang here is in a build script for miniz-sys. Turns out the build script just... never exits. It's not an sccache issue and it's not a the-build-script-executed-clang issue. Presumably it's a build-script-was-linked-by-clang issue, but that seems crazy.

Looking at strace one of the threads looks like it segfaults but doesn't take down the whole process. I'm extremely confused. Trying to narrow down what's happening.

alexcrichton · 2018-05-09T20:55:00Z

So we're not only compiling LLVM with clang but we're also compiling things like jemalloc with clang. The segfault was somewhere deep within jemalloc on an instruction which was moving from memory which I believe required at least 8 byte alignment (maybe 16?) and didn't have that alignment. That's what caused the segfault itself, and as to why the build script wasn't taken down I'm gonna chalk that up to "probably some glibc bug fixed in the last decade".

Presumably clang is just making different assumptions about stack alignment than gcc is? Not entirely sure. In the meantime I'm now testing passing -mstackrealign which will tell clang to have each function realign the stack. Not exactly speedy but hey it's 32-bit linux anyway. If you want speed you're cross-compiling from 64-bit linux.

Currently on CI we predominately compile LLVM with the default system compiler which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang 6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This looks to generate faster code according to rust-lang#49879 which translates to a faster rustc (as LLVM internally is faster) The major changes here were to the containers that build Linux releases, basically adding a new step that uses the previous gcc 4.8 compiler to compile the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been updated to download precompiled versions of Clang 6 and configure the build to use them. Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as well as an update to `sccache` on Windows which was needed to correctly work with `clang-cl`. Finally the MinGW compiler is entirely left out here intentionally as it's currently thought that Clang can't generate C++ code for MinGW and we need to use gcc, but this should be verified eventually.

alexcrichton · 2018-05-09T21:45:46Z

@bors: r=kennytm

Seems to have fixed the issue

bors · 2018-05-09T21:45:47Z

💡 This pull request was already approved, no need to approve it again.

This pull request previously failed. You should add more commits to fix the bug, or use retry to trigger a build again.
There's another pull request that is currently being tested, blocking this pull request: Make SelectionCache and EvaluationCache thread-safe #49834

bors · 2018-05-09T21:45:47Z

📌 Commit 61e1f23 has been approved by kennytm

alexcrichton · 2018-05-09T21:45:57Z

@bors: r=kennytm

bors · 2018-05-09T21:45:57Z

📌 Commit 7e5b9ac has been approved by kennytm

bors · 2018-05-09T21:52:44Z

⌛ Testing commit 7e5b9ac with merge 04d50c76ea59e11a0dcd8ac6a36d8be47ad29614...

bors · 2018-05-10T00:53:20Z

💔 Test failed - status-travis

alexcrichton · 2018-05-10T03:18:53Z

@bors: retry

No major slowdown from the previous successful run:

-  10.184 (  89.036 =>   78.852) Std { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu" }
-  12.257 (  62.006 =>   49.749) Docs { stage: 2, host: "i686-unknown-linux-gnu" }
+  13.271 ( 115.781 =>  129.052) Llvm { target: "i686-unknown-linux-gnu", emscripten: true }
-  13.744 (  79.546 =>   65.802) CodegenBackend { compiler: Compiler { stage: 1, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", backend: "emscripten" }
-  15.592 (  86.858 =>   71.266) CodegenBackend { compiler: Compiler { stage: 1, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", backend: "llvm" }
-  18.188 ( 168.000 =>  149.812) Rustc { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" } }
-  19.101 ( 252.878 =>  233.777) ToolBuild { compiler: Compiler { stage: 0, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", tool: "rustbook", path: "src/tools/rustbook", mode: Librustc, is_ext_tool: false, extra_features: [] }
-  29.137 ( 142.258 =>  113.121) Rustdoc { host: "i686-unknown-linux-gnu" }
+  32.046 ( 248.280 =>  280.326) Llvm { target: "i686-unknown-linux-gnu", emscripten: false }
-  50.573 ( 253.241 =>  202.668) ToolBuild { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", tool: "rustfmt", path: "src/tools/rustfmt", mode: Librustc, is_ext_tool: true, extra_features: [] }
-  51.114 ( 244.849 =>  193.735) ToolBuild { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", tool: "clippy-driver", path: "src/tools/clippy", mode: Librustc, is_ext_tool: true, extra_features: [] }
-  76.286 ( 414.507 =>  338.221) ToolBuild { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", tool: "cargo", path: "src/tools/cargo", mode: Librustc, is_ext_tool: false, extra_features: [] }
-  77.728 (1076.627 =>  998.899) Rustc { target: "i686-unknown-linux-gnu", compiler: Compiler { stage: 0, host: "i686-unknown-linux-gnu" } }
-  80.869 ( 383.886 =>  303.017) ToolBuild { compiler: Compiler { stage: 2, host: "i686-unknown-linux-gnu" }, target: "i686-unknown-linux-gnu", tool: "rls", path: "src/tools/rls", mode: Librustc, is_ext_tool: true, extra_features: ["clippy"] }
- 218.144 (1203.306 =>  985.162) Rustc { target: "i686-unknown-linux-gnu", compiler: Compiler { stage: 1, host: "i686-unknown-linux-gnu" } }
- 358.084 ( 358.084 =>    0.000) Extended { stage: 2, host: "i686-unknown-linux-gnu", target: "i686-unknown-linux-gnu" }
- 432.515 ( 432.515 =>    0.000) PlainSourceTarball

otherwise I think the new docker image just took awhile to build

bors · 2018-05-10T04:43:06Z

⌛ Testing commit 7e5b9ac with merge 2a1af97...

bors · 2018-05-10T07:43:39Z

💔 Test failed - status-travis

kennytm · 2018-05-10T09:34:14Z

@bors retry

3 hour timeout.

Step	Duration
stage1-rustc	37min
stage0-rustc	24min
run-pass	14min
run-pass-fulldeps	9min
plain source tarball	8min
test/core	6min
llvm	6min
another llvm 🤔	5min
test/std	5min
stage0-rustbook	5min

bors · 2018-05-10T10:30:09Z

⌛ Testing commit 7e5b9ac with merge 57dc984...

bors · 2018-05-10T12:47:40Z

☀️ Test successful - status-appveyor, status-travis
Approved by: kennytm
Pushing 57dc984 to master...

michaelwoerister · 2018-05-10T13:21:02Z

🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉 (one for each failed bors run)

alexcrichton · 2018-05-10T14:01:34Z

omg it worked!

michaelwoerister · 2018-05-10T21:59:56Z

The results for small crates are pretty spectacular and for larger crates they are rather good too:
http://perf.rust-lang.org/compare.html?start=95d0b9e96795aea20ac4a1fad9251982714d3c55&end=0a223d139cd26e5bfab23a478a5cad845eaab131&stat=wall-time

Thanks for all the work all of you have put into this PR!

scottmcm · 2018-05-11T04:47:41Z

Wow! That's an impressive amount of green!