Skip to content

Travis CI SDE testing#124

Closed
jeffhammond wants to merge 8 commits intoflame:masterfrom
jeffhammond:travis-ci-sde-knl-testing
Closed

Travis CI SDE testing#124
jeffhammond wants to merge 8 commits intoflame:masterfrom
jeffhammond:travis-ci-sde-knl-testing

Conversation

@jeffhammond
Copy link
Member

This pull request implements #122.

This overhauls most of the Travis CI stuff. In particular:

As noted by @devinamatthews, performance with SDE is ~0.02 GF/s, so we probably need to prune the tests if we want them to finish in under an hour.

@jeffhammond jeffhammond self-assigned this May 4, 2017
@jeffhammond jeffhammond mentioned this pull request May 4, 2017
Closed
Copy link
Member

@devinamatthews devinamatthews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For tests that require SDE, perhaps we should cut the tests back to:

  1. gemm, trsm, and gemmtrsm ukernels (including all storage types)
  2. Ideally, packing ukernels in isolation but a small gemm should suffice.
  3. Unaligned C, could be combined with 1) and/or 2).
  4. The above for all datatypes, including 1m but not 3m/4m.

@devinamatthews
Copy link
Member

Also note that the travis_driver.sh script checks /proc/cpuinfo which won't work on OSX.

@jeffhammond
Copy link
Member Author

I properly guarded access to /proc/cpuinfo in https://github.com/jeffhammond/blis/blob/travis-ci-sde-knl-testing/.travis.yml#L66. I forgot to add the guard in https://github.com/jeffhammond/blis/blob/travis-ci-sde-knl-testing/build/travis_driver.sh because Travis CI only tests auto on Mac and thus those statements will only be executed on Linux.

I will fix this.

@devinamatthews
Copy link
Member

devinamatthews commented May 4, 2017

How about these test input files (for tests that run with SDE)?

Archive.zip

@jeffhammond
Copy link
Member Author

@devinamatthews Thanks. I'll incorporate them shortly.

- it is hard to do all the branching required for this in the .travis.yml
  so we drop into travis_driver.sh and handle everything.
- detect hardware and run tests natively when possible
- only use -d sde for KNL (requires merge w/ not-real-knl branch)
- add SDE download+install
- compile tests outside of SDE
- support 32-bit Travis (unlikely?)
@devinamatthews
Copy link
Member

@jeffhammond can you elaborate on ${SDE_LOCATION}?

@jeffhammond
Copy link
Member Author

SDE_LOCATION is a link to my personal Dropbox. SDE requires one to accept license terms to use, and there is no way to enforce that with a direct download, so it was determined that a private link to e.g. Dropbox was the most effective way to get SDE into the Travis environment.

There are likely better methods. I'm happy to accept suggestions.

@jeffhammond
Copy link
Member Author

Now that I think of it, I can make the link public if I password-protect the file and then hide the password in a Travis secure env var, but I have a bad experience with that in the context of doing that with Intel compiler license keys when trying to support Intel compilers in Travis.

@devinamatthews
Copy link
Member

devinamatthews commented May 24, 2018

This has been implemented in the travis-virtualization branch and is almost fully working now. As it turns out, there is a little wrinkle in emulating the AMD architectures because glibc and libm actually do their own CPUID detection and dispatch, and end up executing FMA4 instructions. Believe it or not, the only way to work around this is to apply a binary patch to ld.so!

@jeffhammond
Copy link
Member Author

What about using MUSL instead of GLIBC? MUSL may not have all the CPU-specific optimizations and therefore behave better.

@devinamatthews
Copy link
Member

I've got a workaround in place, now I just have to deal with the kernel security flags so that sde will run.

@devinamatthews
Copy link
Member

Closing in favor of #212, which incorporates all of the contributions here. Thanks @jeffhammond for the idea and the crucial details.

@jeffhammond jeffhammond deleted the travis-ci-sde-knl-testing branch November 6, 2019 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants