Get rid of build shell scripts and move them to Python by mehrdadn · Pull Request #6082 · ray-project/ray

mehrdadn · 2019-11-04T13:03:14Z

Why are these changes needed?

Wheel building on Windows has unexpected behavior when invoked from MSYS2 Bash, as it alters the environment when calling Bazel.
Bazel is aiming to get rid of the MSYS2 dependency on Windows; it's best to stick with Python.

Checks

I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://ray.readthedocs.io/en/latest/.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.

build.py

edoakes

This is a good idea but this script is extremely hard to understand, and maintainability is a top priority. Please see my comments and try to clean it up generally and I will take another look. Also, adding comments that explain the windows-specfic behavior anywhere that it's non-obvious would be great.

build.py

mehrdadn · 2019-11-05T01:40:43Z

This is a good idea but this script is extremely hard to understand, and maintainability is a top priority. Please see my comments and try to clean it up generally and I will take another look. Also, adding comments that explain the windows-specfic behavior anywhere that it's non-obvious would be great.

@edoakes: It actually turns out I thought this script is supposed to be run directly by users, but @pcmoritz mentioned that it's actually only supposed to be run from setup.py, generally via pip. That fundamentally changes what I thought the script is be responsible for, and it means I should take out a lot of the things that don't make sense inside a pip installation (like handling of system packages). I'll move the relevant parts into setup.py and update the PR.

mehrdadn · 2019-11-12T13:22:11Z

@pcmoritz I just moved things to setup.py. Could you let me know (or just fill in yourself) what should be in place of the previous build.sh calls (see the TODOs)?

python/setup.py

java/doc/installation.rst

pcmoritz · 2019-11-12T21:45:54Z

This looks really great, thanks for doing it!

edoakes

Nice, much simpler now. Some more style feedback.

java/doc/installation.rst

python/setup.py

Resolved

mehrdadn · 2019-11-18T06:53:15Z

@pcmoritz @edoakes I think this should be ready for merge, though I'm not entirely sure why the build is failing (it seems unrelated as far as I can tell). Can you verify and merge if it seems correct?

edoakes · 2019-11-19T04:47:29Z

The python tests are unrelated but the linter is failing:
https://travis-ci.com/ray-project/ray/jobs/257550768

You should be able to run ./ci/travis/format.sh to lint locally. You can also run ./setup_hooks.sh to set up a github push hook that will run the linter before pushing code to the remote repo.

python/setup.py

edoakes

Looks pretty good. My chief complaint is that I found download_pickle5 extremely hard to follow - any way to make that a bit more clear at the cost of verbosity?

python/setup.py

AmplabJenkins · 2020-07-15T22:47:42Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28408/
Test FAILed.

AmplabJenkins · 2020-07-16T01:21:40Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28422/
Test FAILed.

mboisson · 2022-08-24T13:54:01Z

This PR made it impossible to build from source using pip:

Collecting https://github.com/ray-project/ray/archive/ray-2.0.0.tar.gz
  Downloading https://github.com/ray-project/ray/archive/ray-2.0.0.tar.gz
     \ 48.6 MB 13.2 MB/s 0:00:03
ERROR: https://github.com/ray-project/ray/archive/ray-2.0.0.tar.gz does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

mehrdadn force-pushed the master branch 5 times, most recently from 80fc5f1 to 003890a Compare November 4, 2019 19:44

pcmoritz reviewed Nov 4, 2019

View reviewed changes