Proposal: Providing a Consistent CI/CD Experience #398
Description
Introduction
Across all OpenTelemetry repositories there is currently 5 different, active CI providers. Each of these providers have their own way of executing tests, interacting with the user, and publishing test results. This can make it difficult for newcomers to contribute to multiple OpenTelemetry repositories.
Current Landscape
| Repository | CI Provider | Automated Build and Test | Code Coverage | Automated Performance Testing | Automated Deployment | Automated Docs Deployment |
|---|---|---|---|---|---|---|
| Collector | CircleCI | [x] | [x] | [x] | [x] | [] |
| C++ | GHA | [x] | [x] | [x] | [] | [] |
| JavaScript | CircleCI/GHA | [x] | [x] | [] | [] | [x] |
| .NET | Azure | [x] | [x] | [] | [] | [] |
| PHP | Travis | [x] | [x] | [] | [] | [] |
| Java | CircleCI | [x] | [x] | [x] | [x] | [x] |
| Python | Travis/CircleCI | [x] | [x] | [] | [x] | [x] |
| Ruby | CircleCI | [x] | [] | [] | [x] | [] |
| Go | CircleCI | [x] | [x] | [] | [] | [] |
| Swift | GHA/Scope | [x] | [] | [] | [] | [] |
| Rust | CircleCI | [x] | [x] | [] | [] | [x] |
| Erlang | CircleCI | [x] | [x] | [x] | [] | [] |
Proposal
I propose that all languages consider using the same CI provider. This would create a more consistent development process and make it easier for developers to contribute to multiple language libraries.
We suggest that provider be GitHub Actions. Here’s why:
Ease-of-Use
CircleCI and Travis will automatically run when pull requests and commits are issued against the repository. But if a contributor forks the repository, unless they set up an account with the CI provider and link it to their forked repository, CI will not be activated and tests will not be run automatically.
In contrast, GitHub Actions works out of the box on a forked repository and can be easily configured to run a test workflow each time a commit is issued. This would help individual contributors test their code and ensure code quality before submitting a pull request against the repository.
Transparency
Current CI providers such as CircleCI and Travis allow anyone to view the console output when building and running tests but the test results can not be seen anywhere on the GitHub repository. To view this testing output: You need go to a different website, navigate a different user interface, and then sift through thousands of lines of console output. This is not a seamless developer experience.
In contrast, using GitHub Actions would provide all testing output directly on the repository’s GitHub page, which would help contributors to find, read, and use the test output to maintain code quality.
Control
GitHub Actions’ integration with other GitHub features means you can have finer control over the CI pipeline. For example, certain workflows can be set to only run on a new release. Workflows can even be used to close stale issues and pull requests.
Recommendation
I recommend that we consider using one consistent CI provider, GitHub Actions, which provides an integrated and seamless developer experience for all contributors.
Example
Please see this example that the C++ repository has adopted for the above reasons.