Skip to content

[CI] Fix-Up Docker Build (Use Python)#11139

Merged
simon-mo merged 31 commits intoray-project:masterfrom
ijrsvt:docker-build-cleanup
Oct 12, 2020
Merged

[CI] Fix-Up Docker Build (Use Python)#11139
simon-mo merged 31 commits intoray-project:masterfrom
ijrsvt:docker-build-cleanup

Conversation

@ijrsvt
Copy link
Copy Markdown
Contributor

@ijrsvt ijrsvt commented Sep 30, 2020

Why are these changes needed?

Our current CI Docker building solution is a bash nightmare--this moves it into python so it is more maintainable and reliable.
It also fixes our logic that rebuilds all images when a PR has a non-standard branch name. This also allows us to rebuild & push base images when dependencies change (as opposed to just every 2 weeks).

The new output looks like: this

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@simon-mo
Copy link
Copy Markdown
Contributor

simon-mo commented Oct 2, 2020

do you think it make sense to use docker's python sdk? https://docker-py.readthedocs.io/en/stable/images.html#docker.models.images.ImageCollection.build

ijrsvt and others added 5 commits October 5, 2020 14:44
Co-authored-by: Sumanth Ratna <sumanthratna@gmail.com>
Co-authored-by: Sumanth Ratna <sumanthratna@gmail.com>
@ijrsvt ijrsvt changed the title [WIP] Fix-Up Docker Build [CI] Fix-Up Docker Build (Use Python) Oct 8, 2020
Copy link
Copy Markdown
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks amazing!!!

@simon-mo simon-mo merged commit 6426fb3 into ray-project:master Oct 12, 2020
Copy link
Copy Markdown
Member

@sumanthratna sumanthratna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijrsvt this looks really nice! I left some post-merge comments on very minor things to change in a later PR. if you want I can push a PR to fix these comments next weekend

@ijrsvt ijrsvt deleted the docker-build-cleanup branch October 12, 2020 23:24
@ijrsvt
Copy link
Copy Markdown
Contributor Author

ijrsvt commented Oct 12, 2020

@ijrsvt this looks really nice! I left some post-merge comments on very minor things to change in a later PR. if you want I can push a PR to fix these comments next weekend

Thanks so much for the feedback--if you could make a PR with some of the fixes above that would be fantastic! Thanks for taking the time @sumanthratna

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants