Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Update framework coverage difference commenter #14517

Merged

Conversation

jcogs33
Copy link
Contributor

@jcogs33 jcogs33 commented Oct 16, 2023

Description

The framework coverage difference commenter currently posts a new comment after new commits. When there are a lot of these comments on a PR, it can be hard to find comments from reviewers. This PR changes the workflow so that it updates one comment box instead of creating multiple comments.

Note: this code is based on the QHelp preview comment workflow which updates one comment box. The first commit 9263cfd is based on this code in qhelp-pr-preview.yml. The second commit 6e29b70 is based on this code in post-pr-comment.yml.

This PR also fixes a bug ee4a9c3 that was causing a 'list index out of range' error in the pre-existing get_previous_run_id function. f"branch='{pr_branch}'" resulted in incorrect branch names such as 'branch='"'"'test-commenting'"'"'' due to apparent escaping of the single quotes, and the subprocess_check_output call always returned an empty list as a result since the branch name was incorrect. Changing this code to f"branch={pr_branch}" seems to resolve the issue.

Testing

I've tested these changes on https://github.com/dsp-testing/codeql-framework-coverage-pr-commenter/pull/27 (let me know if you don't have access to this PR).

  • The first commit removes a MaD model, so the workflow posts an initial comment (workflow run).
  • The second commit removes another MaD model, so the workflow updates the existing comment (workflow run).
  • The third commit tests the bug fix in the get_previous_run_id function. It does not change any MaD models, so the diff is the same as the prior run, and the comment is not updated (workflow run).

@jcogs33 jcogs33 changed the title Update framework coverage difference commenter CI: Update framework coverage difference commenter Oct 16, 2023
@jcogs33 jcogs33 force-pushed the jcogs33/update-framework-cov-diff-workflow branch from af7ad50 to ee4a9c3 Compare October 19, 2023 23:24
@jcogs33 jcogs33 added the no-change-note-required This PR does not need a change note label Oct 20, 2023
@jcogs33 jcogs33 marked this pull request as ready for review October 20, 2023 18:03
@jcogs33 jcogs33 requested a review from a team as a code owner October 20, 2023 18:03
Copy link
Collaborator

@adityasharad adityasharad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Some minor suggestions to make the script more robust -- let me know if you want to talk them through in more detail.

misc/scripts/library-coverage/comment-pr.py Show resolved Hide resolved
misc/scripts/library-coverage/comment-pr.py Show resolved Hide resolved
misc/scripts/library-coverage/comment-pr.py Outdated Show resolved Hide resolved
.github/workflows/csv-coverage-pr-artifacts.yml Outdated Show resolved Hide resolved
adityasharad
adityasharad previously approved these changes Oct 24, 2023
Copy link
Collaborator

@adityasharad adityasharad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Optional suggestion only.

misc/scripts/library-coverage/comment-pr.py Outdated Show resolved Hide resolved
misc/scripts/library-coverage/comment-pr.py Show resolved Hide resolved
Co-authored-by: Aditya Sharad <6874315+adityasharad@users.noreply.github.com>
@jcogs33
Copy link
Contributor Author

jcogs33 commented Oct 25, 2023

Thanks @adityasharad! I've applied your suggestion, can you re-approve when you have a chance? 🙏

@jcogs33 jcogs33 merged commit c7b9e40 into github:main Oct 25, 2023
7 checks passed
@jcogs33 jcogs33 deleted the jcogs33/update-framework-cov-diff-workflow branch October 25, 2023 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-change-note-required This PR does not need a change note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants