CI: add the sanitizers pipelines (e.g. ASAN) to Buildkite by tkf · Pull Request #41530 · JuliaLang/julia

tkf · 2021-07-09T23:48:47Z

This is a continuation of #41522 and simply "codifies" what is already documented.

@staticfloat @DilumAluthge I mostly just copied llvmpasses. Let me know if I need to fix something.

DilumAluthge · 2021-07-10T01:24:58Z

Instead of trying to write stuff into /tmp, can you just do make in-place, like we do in the llvm passes?

tkf · 2021-07-10T01:35:07Z

Hmm... I kinda want to keep the out-of-tree -based script so that it is easy to try it out. It's also nice to have a separated toolchain directory so that you don't need to re-install the toolchain (e.g., we can have contrib/tsan.sh that shares the toolchain with contrib/asan.sh).

Maybe we can use ./tmp (which is already listed in .gitignore)?

DilumAluthge · 2021-07-10T01:51:49Z

Maybe we can use ./tmp (which is already listed in .gitignore)?

Sure, let's give that a try.

tkf · 2021-07-10T03:23:42Z

Too long? (Interesting that it's just two seconds less than the time limit)

.buildkite/sanitizers.yml

DilumAluthge · 2021-07-10T03:46:06Z

Too long? (Interesting that it's just two seconds less than the time limit)

We should probably increase the timeout for that specific pipeline to be 120 minutes.

DilumAluthge · 2021-07-10T05:33:15Z

It would be good to edit those echo commands to be a little more descriptive. For example, what does contrib/asan.sh do, and what does contrib/check-asan.jl do?

DilumAluthge · 2021-07-10T09:20:24Z

LGTM.

vchuravy · 2021-07-10T16:27:55Z

One issue here is that the LLVM used is not built with ASAN itself, normally we recommend USE_BINARYBUILDER_LLVM=0 for the second stage to build LLVM itself with ASAN, but that might be to expensive to do on CI and we would need to maintain a binary cache (like for the assert build)

tkf · 2021-07-10T21:00:39Z

I think it's still nice to make sure our codebase itself is ASAN compatible; i.e., the code path with JL_ASAN_ENABLED is compilable and there is no apparent problem in the run-time that can be detected with ASAN.

contrib/asan/Make.user.asan

* Setup CI for ASAN * Launch the `sanitizers.yml` unsigned pipeline * Use a workspace directory in ./tmp * Add some log group headers to make the logs easier to navigate * Install `julia` binary inside sandbox * Double timeout * More descriptive message from sanitizer CI * Fix the path to the binary * Use addenv * Apply suggestions from code review Co-authored-by: Elliot Saba <staticfloat@gmail.com> * Group ASAN related files under contrib/asan/ * Remove redundant JULIA_PRECOMPILE=1 Co-authored-by: Dilum Aluthge <dilum@aluthge.com> Co-authored-by: Elliot Saba <staticfloat@gmail.com> (cherry picked from commit 84934e6)

tkf · 2021-07-19T17:17:20Z

Maybe better not backporting this?
#41554 (comment)
#41499 (comment)

DilumAluthge · 2021-07-20T09:22:14Z

This is my thinking: the CI PRs that we've been making tend to build on one another. So it will probably be cleanest just to backport all CI PRs.

Once I merge #41606, it can be backported. One of the things that #41606 does is that it only runs asan on master. So if we backport #41606 to 1.6, then asan won't run on the release-1.6 branch, which I think is the correct behavior.

tkf · 2021-07-20T19:56:46Z

Thanks for the explanation. I saw your comments in #41554 (comment) too and I see that it makes sense either way.

* Setup CI for ASAN * Launch the `sanitizers.yml` unsigned pipeline * Use a workspace directory in ./tmp * Add some log group headers to make the logs easier to navigate * Install `julia` binary inside sandbox * Double timeout * More descriptive message from sanitizer CI * Fix the path to the binary * Use addenv * Apply suggestions from code review Co-authored-by: Elliot Saba <staticfloat@gmail.com> * Group ASAN related files under contrib/asan/ * Remove redundant JULIA_PRECOMPILE=1 Co-authored-by: Dilum Aluthge <dilum@aluthge.com> Co-authored-by: Elliot Saba <staticfloat@gmail.com> (cherry picked from commit 84934e6)

Setup CI for ASAN

cb70c26

tkf requested a review from a team as a code owner July 9, 2021 23:48

DilumAluthge requested review from DilumAluthge and staticfloat and removed request for a team July 10, 2021 01:15

DilumAluthge added the ci Continuous integration label Jul 10, 2021

DilumAluthge changed the title ~~Setup CI for ASAN~~ CI: add the sanitizers pipelines (e.g. ASAN) to Buildkite Jul 10, 2021

Launch the sanitizers.yml unsigned pipeline

0d28864

Use a workspace directory in ./tmp

ff9c978

Add some log group headers to make the logs easier to navigate

f462fb1

tkf commented Jul 10, 2021

View reviewed changes

.buildkite/sanitizers.yml Show resolved Hide resolved

tkf commented Jul 10, 2021

View reviewed changes

.buildkite/sanitizers.yml Outdated Show resolved Hide resolved

tkf added 2 commits July 10, 2021 01:25

Install julia binary inside sandbox

61c77cb

Double timeout

dc6408e

tkf added 2 commits July 10, 2021 01:53

More descriptive message from sanitizer CI

0b004ab

Fix the path to the binary

cd295fe

DilumAluthge mentioned this pull request Jul 10, 2021

CI: Issue to track which CI jobs have been migrated to Buildkite #41509

Closed

34 tasks

Use addenv

eee4cf3

DilumAluthge approved these changes Jul 10, 2021

View reviewed changes

DilumAluthge added the building Build system, or building Julia or its dependencies label Jul 10, 2021

vchuravy requested a review from maleadt July 10, 2021 16:28

Group ASAN related files under contrib/asan/

32d436f

staticfloat approved these changes Jul 13, 2021

View reviewed changes

contrib/asan/Make.user.asan Outdated Show resolved Hide resolved

Remove redundant JULIA_PRECOMPILE=1

cde6cef

DilumAluthge added merge me PR is reviewed. Merge when all tests are passing backport 1.6 Change should be backported to release-1.6 backport 1.7 labels Jul 13, 2021

staticfloat merged commit 84934e6 into JuliaLang:master Jul 13, 2021

DilumAluthge removed the merge me PR is reviewed. Merge when all tests are passing label Jul 13, 2021

tkf deleted the ci-asan branch July 13, 2021 23:54

KristofferC mentioned this pull request Jul 19, 2021

Backports for julia-1.6.3 #41554

Merged

75 tasks

KristofferC mentioned this pull request Jul 19, 2021

release-1.7: Backports for 1.7-RC1/1.7-beta4 #41499

Merged

82 tasks

tkf removed backport 1.6 Change should be backported to release-1.6 backport 1.7 labels Jul 19, 2021

DilumAluthge added backport 1.6 Change should be backported to release-1.6 backport 1.7 labels Jul 20, 2021

KristofferC removed the backport 1.7 label Aug 3, 2021

KristofferC mentioned this pull request Aug 25, 2021

release-1.7: Backports for 1.7-RC1 #41781

Merged

63 tasks

KristofferC removed the backport 1.6 Change should be backported to release-1.6 label Sep 7, 2021

tkf mentioned this pull request Oct 1, 2021

CI: add TSAN to the sanitizers pipelines #42444

Merged

Uh oh!

Conversation

tkf commented Jul 9, 2021

Uh oh!

DilumAluthge commented Jul 10, 2021

Uh oh!

tkf commented Jul 10, 2021

Uh oh!

DilumAluthge commented Jul 10, 2021

Uh oh!

tkf commented Jul 10, 2021

Uh oh!

Uh oh!

Uh oh!

DilumAluthge commented Jul 10, 2021

Uh oh!

DilumAluthge commented Jul 10, 2021

Uh oh!

DilumAluthge commented Jul 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vchuravy commented Jul 10, 2021

Uh oh!

tkf commented Jul 10, 2021

Uh oh!

Uh oh!

tkf commented Jul 19, 2021

Uh oh!

DilumAluthge commented Jul 20, 2021

Uh oh!

tkf commented Jul 20, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

DilumAluthge commented Jul 10, 2021 •

edited

Loading