Skip to content

Auto-cancel build when a new commit is pushed#8043

Merged
simon-mo merged 21 commits intoray-project:masterfrom
mehrdadn:auto-cancel-ci
Jul 23, 2020
Merged

Auto-cancel build when a new commit is pushed#8043
simon-mo merged 21 commits intoray-project:masterfrom
mehrdadn:auto-cancel-ci

Conversation

@mehrdadn
Copy link
Copy Markdown
Contributor

@mehrdadn mehrdadn commented Apr 16, 2020

This allows builds to self-terminate when they detect new commits on the remote.

For cases where this is intentional and desirable, this mechanism can be bypassed by keeping the remote branch alive, and putting a line like the following in the commit message:

CI_KEEP_ALIVE

A list of specific CI providers to keep alive can also be provided as follows:

CI_KEEP_ALIVE: travis, github

Why are these changes needed?

Builds currently don't stop when remote branches are overwritten.

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/latest/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failure rates at https://ray-travis-tracker.herokuapp.com/.
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested (please justify below)

@mehrdadn mehrdadn force-pushed the auto-cancel-ci branch 30 times, most recently from ad3d480 to 6e97e8b Compare April 16, 2020 14:48
@mehrdadn mehrdadn requested a review from simon-mo July 21, 2020 22:35
Comment on lines +75 to +77
head = git("symbolic-ref", "-q", "HEAD")
ref = git("for-each-ref", "--format=%(upstream:short)", head)
except subprocess.CalledProcessError:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is leaky. when calling git, how do we know CalledProcessErorr will happen? Consider handle this in git implementation by catching something like this and return None?

Copy link
Copy Markdown
Contributor Author

@mehrdadn mehrdadn Jul 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I'm confused what you mean here. CalledProcessError happens when git exits with a nonzero status. We need the status code in some cases (though not here) so I can't just return None. I'm not sure I understand what the problem is though -- this is the normal way to indicate a subprocess call was unsuccessful. What's leaky?

Copy link
Copy Markdown
Contributor

@simon-mo simon-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks much better now! Can you write some description about how exactly does this script determines the new commits. In particular, explain how the following works together because they may be foreign to developers:

  • git symbolic-ref
  • git for-each-ref
  • git ls-remote and the assumptions that ref/pulls or ref/heads will contains the pull request number.


def main(program, *args):
p = argparse.ArgumentParser()
p.add_argument("--skip_repo", action="append", help="Repo to exclude.")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we hard code this as a constant in this file?

Copy link
Copy Markdown
Contributor Author

@mehrdadn mehrdadn Jul 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying not to make the script specific to ray-project/ray... it might be useful elsewhere?

return None


def git_branch_info():
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this function simpler?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try again, but it might take a while... I recall it was originally simple and I had to complicate it to handle all the different cases.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I know how to simplify this :\ I recall it was extremely painful getting all the edge cases right... Travis vs. GitHub, merge commit vs. normal commit, single commit vs. multiple commits, etc.

@mehrdadn
Copy link
Copy Markdown
Contributor Author

@AmplabJenkins
Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28873/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28875/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28878/
Test FAILed.

@AmplabJenkins
Copy link
Copy Markdown

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/28879/
Test FAILed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants