Skip to content

Update GitHub Actions#974

Merged
nucleogenic merged 1 commit intodevelopfrom
nucleogenic-update-github-actions
Nov 18, 2022
Merged

Update GitHub Actions#974
nucleogenic merged 1 commit intodevelopfrom
nucleogenic-update-github-actions

Conversation

@nucleogenic
Copy link
Copy Markdown
Member

@nucleogenic nucleogenic commented Nov 8, 2022

Introduction

This change set contains proposed changes to the GitHub Actions workflows for the project.

Changes

  • Added web UI workflow (currently runs flake8 and black for Python; stylelint and prettier for CSS)
  • Fixed broken/false positive C++ unit test execution step
  • Split SonarCloud analysis into a separate job (allows unit tests to run independently/in parallel)
  • Prevented SonarCloud job from running on forks
  • Added cache for SonarCloud installation
  • Added cache for SonarCloud scanning (branch-level cache for now)
  • Added parallel build jobs for ARM cross-compilation

Execution Behaviour

  • Event: pull_request
    • Run ARM build workflow and C++ workflow only if cpp/** changed
    • Run web UI workflow only if python/** changed

(Note: the paths in on->pull_request->paths are checked against all commits in the PR, not just the latest push.)

Future Improvements

  • Web UI workflow
    • Run web UI test suite
    • Add SonarCloud scan against Python/HTML/CSS sources
  • SonarCloud cache
    • Build cache against develop branch to speed up first execution on branches off develop

⚠️ Todo Before Merge

  • Replace @nucleogenic-update-github-actions with @develop in build_code.yml

@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic Note that the sonar.cfamily.cache.enabled option is deprecated and will be removed with the next major SonarQube release. The SonarQube documentation contains the details.

@nucleogenic
Copy link
Copy Markdown
Member Author

@nucleogenic Note that the sonar.cfamily.cache.enabled option is deprecated and will be removed with the next major SonarQube release. The SonarQube documentation contains the details.

Hi @uweseimet, thanks for making me aware.

I can see the deprecation notice in the SonarQube docs:
https://docs.sonarqube.org/latest/analysis/languages/cfamily/

Looking at the SonarCloud docs, this doesn't seem to be deprecated yet?
https://docs.sonarcloud.io/advanced-setup/languages/c-c-objective-c/

I can't find any reference to the new properties being available for SonarCloud.

@nucleogenic nucleogenic force-pushed the nucleogenic-update-github-actions branch from 3af406d to b0e9455 Compare November 10, 2022 09:02
@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic As far as I can tell SonarCloud is regularly updated with the latest SonarQube version. Since SonarCloud is a hosted SonarQube instance as far as properties are concerned they are those also supported by SonarQube.
Saying that, I expect it to be just a matter of time until changes in SonarQube become relevant for SonarCloud. The cache behavior/option will change in the next major SonarQube version, that's why this property is still available in SonarCloud.

@nucleogenic
Copy link
Copy Markdown
Member Author

I tried introducing the new properties, but they were ignored.

The scanner gives this output:

to enable cache please specify the following 2 options:
sonar.cfamily.cache.enabled=true
sonar.cfamily.cache.path=relative_or_absolute_path_to_cache_location

* visit the documentation page for more information
https://docs.sonarcloud.io/advanced-setup/languages/c-c-objective-c/

(https://github.com/akuker/RASCSI/actions/runs/3435586763/jobs/5728119135)

I also checked to see if we are using the latest scanner version, which we seem to be:

SONAR_SCANNER_VERSION: 4.7.0.2747

https://docs.sonarcloud.io/advanced-setup/ci-based-analysis/sonarscanner-cli/
https://docs.sonarqube.org/latest/analysis/scan/sonarscanner/

I am not familiar enough with the SonarSource solutions to provide any further insight beyond we're not getting deprecation warnings at the moment. Regardless, it will be an easy change to make in the future.

@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic I'm not sure whether I correctly understand: The respective option can become ineffective any time. When this happens, what is going to be the impact? If there is any impact, how will it be resolved, i.e. what is the easy change?

@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic One more question regarding "Fixed broken/false positive C++ unit test execution step". What was broken here? I'm asking because I have not noticed anything that does not work with the C++ unit tests and the C++ code analysis.

@nucleogenic
Copy link
Copy Markdown
Member Author

nucleogenic commented Nov 10, 2022

@uweseimet My current understanding is that the properties are not deprecated in SonarCloud yet. Can you provide a source to suggest otherwise? I can't find one.

The properties can be renamed easily in the future, of course, when they are supported.

Let me know if you have any concerns?

@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic I would like to learn what is going to happen once these properties are not supported anymore. This can happen any time with a SonarCloud maintenance update. What will be the impact on the github action, and what has to be done in order to fix that? What does have to be renamed how?
If properties are added and it is already known at the time of adding them that they are going to be removed it is important to know how to address issues their removal will cause.

@nucleogenic
Copy link
Copy Markdown
Member Author

@nucleogenic I would like to learn what is going to happen once these properties are not supported anymore. This can happen any time with a SonarCloud maintenance update.

The new properties don't appear to be supported via SonarCloud yet, so it doesn't appear to be an option for us to update them right now. You could apply this argument to any feature of SonarCloud, of course.

What will be the impact on the github action, and what has to be done in order to fix that? What does have to be renamed how?

The change would be:

Remove:
--define sonar.cfamily.cache.enabled=true
--define sonar.cfamily.cache.path="$HOME/.sonar_cache/"

Add:
--define sonar.cfamily.analysisCache.mode=fs
--define sonar.cfamily.analysisCache.path="$HOME/.sonar_cache/"

Interestingly, the docs state SonarQube defaults to sonar.cfamily.analysisCache.mode=server, which might reduce some of the cache management burden from us, if this is supported via SonarCloud.

If properties are added and it is already known at the time of adding them that they are going to be removed it is important to know how to address issues their removal will cause.

There isn't a deprecation notice - as far as I can see - in the SonarCloud documentation. Since SonarQube is in practice a separate offering, I think we can only make decisions based on the SonarCloud docs.

@nucleogenic
Copy link
Copy Markdown
Member Author

@nucleogenic One more question regarding "Fixed broken/false positive C++ unit test execution step". What was broken here? I'm asking because I have not noticed anything that does not work with the C++ unit tests and the C++ code analysis.

See here (before):
https://github.com/akuker/RASCSI/actions/runs/3436315899/jobs/5729699222

image

Vs here (after):
https://github.com/akuker/RASCSI/actions/runs/3435587007/jobs/5728119583

image

When the command returned a non-zero exit code (e.g. command error, failing tests), GitHub Actions still reported the step as successful, because tee always returned a zero exit code.

@uweseimet
Copy link
Copy Markdown
Contributor

@nucleogenic Thanks a lot for elaborating on the SonarQube property issue and on the C++ analysis issue!

Regarding SonarCloud, it is essentially a SonarQube instance, so anything related to SonarQube will also become relevant for SonarCloud, SonarCloud cannot be that different from SonarQube, because they are just running SonarQube in a special environment. This is quite similar to the relationship of Jenkins to CloudBees, for instance, the latter offering a multi-team solution on top of Jenkins.

@nucleogenic nucleogenic force-pushed the nucleogenic-update-github-actions branch 2 times, most recently from 386bcc4 to 9bd0f9d Compare November 15, 2022 01:42
@nucleogenic nucleogenic marked this pull request as ready for review November 15, 2022 03:49
@nucleogenic nucleogenic force-pushed the nucleogenic-update-github-actions branch from 23bcbe8 to be7bcff Compare November 16, 2022 15:50
…ud cache, introduce parallel execution where possible
@nucleogenic nucleogenic force-pushed the nucleogenic-update-github-actions branch from be7bcff to b481f74 Compare November 16, 2022 15:54
@nucleogenic nucleogenic requested a review from akuker November 16, 2022 15:55
@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@nucleogenic
Copy link
Copy Markdown
Member Author

Hi @akuker @uweseimet @rdmark, I am going to merge this PR now. If there are problems with the GitHub Actions workflows following these changes, please let me know and I will review ASAP.

@nucleogenic nucleogenic merged commit 163cb5a into develop Nov 18, 2022
@uweseimet
Copy link
Copy Markdown
Contributor

uweseimet commented Nov 18, 2022

@nucleogenic There appears to be a problem, indeed: Github Actions does not show any running workflow after the merge. I would have expected the C++ build and code analysis to be run for the develop branch, just like before.

@nucleogenic
Copy link
Copy Markdown
Member Author

Thanks @uweseimet, I agree. I've raised a draft PR for this, but will test on my fork to keep the PR history here clean.

@rdmark rdmark deleted the nucleogenic-update-github-actions branch November 20, 2022 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants