ci: enable coverage by v1v · Pull Request #377 · elastic/elastic-agent

v1v · 2022-04-26T13:53:34Z

What does this PR do?

Report Test Coverage in the CI

Why is it important?

Help to visualise when the Test coverage gets increased or decreased (see #377 (comment))

Questions

Current implementation does run coverage for each platform (windows, linux and ARM), therefore the coverage results are not aggregated, I've seen https://github.com/elastic/beats/blob/main/dev-tools/aggregate_coverage.py.

Is it required to aggregate the results from all platforms?
Does the existing coverage in https://fleet-ci.elastic.co/job/elastic-agent/job/elastic-agent-mbp/job/PR-377/4/coverage/ reflect the real test coverage?

Tasks

Add cov to xml transformation in the Magefile
Converge coverage results? It seems https://github.com/elastic/beats/blob/main/dev-tools/aggregate_coverage.py was in place but was not migrated to this project.

mergify · 2022-04-26T13:53:59Z

This pull request does not have a backport label. Could you fix it @v1v? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2022-04-26T13:59:32Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-04-26T19:29:00.460+0000
Duration: 40 min 52 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3849
Skipped	21
Total	3870

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

mdelapenya · 2022-04-26T14:24:21Z

go.sum

 github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
 github.com/bonitoo-io/go-sql-bigquery v0.3.4-1.4.0/go.mod h1:J4Y6YJm0qTWB9aFziB7cPeSyc6dOZFyJdteSeybVpXQ=
 github.com/boumenot/gocover-cobertura v1.2.0/go.mod h1:fz7ly8dslE42VRR5ZWLt2OHGDHjkTiA2oNvKgJEjLT0=
+github.com/boumenot/gocover-cobertura v1.2.0/go.mod h1:fz7ly8dslE42VRR5ZWLt2OHGDHjkTiA2oNvKgJEjLT0=


I'd also add a reference to tools.go file (on mobile, will paste link asap)

https://github.com/elastic/elastic-agent/blob/main/tools/tools.go

Btw, lines 268 and 269 are exactly the same. Can you run go mod tidy and verify that?

elasticmachine · 2022-04-26T16:54:41Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	95.652% (`66/69`)	❕
Files	69.231% (`144/208`)	❕
Classes	69.048% (`290/420`)	❕
Methods	52.45% (`792/1510`)	❕
Lines	38.789% (`8529/21988`)	❕
Conditionals	100.0% (`0/0`)	💚

v1v · 2022-04-26T18:47:08Z

.ci/Jenkinsfile

            post {
              always {
                junit(allowEmptyResults: true, keepLongStdio: true, testResults: "${BASE_DIR}/build/TEST-*.xml")
+                coverageReport(baseDir: "**/build", reportFiles: 'TEST-go-unit.html', coverageFiles: 'TEST-go-unit-cov.xml')


Question: is it required to aggregate the results from all the different platforms (windows, linux, arm)? If so, https://github.com/elastic/beats/blob/main/dev-tools/aggregate_coverage.py is the one in charge?

jlind23

Having only the Linux coverage can be a good first step.

ph · 2022-04-26T19:26:33Z

I am fine with only Linux, @jlind23 We should probably exclude dev-tools/* WDYT?

v1v · 2022-04-26T19:29:16Z

Having only the Linux coverage can be a good first step.

Thanks, 499e299 is the one only driving the test_coverage for linux

jlind23 · 2022-04-27T12:44:40Z

@ph agree, let's remove it.

v1v · 2022-04-27T14:02:05Z

@ph agree, let's remove it.

How? I could strip those entries in the generated cov file, but I'm not sure if the go coverage supports something natively, any hints?

ph · 2022-04-27T19:12:35Z

@v1v I've look at that, seems not a simple solution other than removing the files or path from the coverage output https://stackoverflow.com/questions/50065448/how-to-ignore-generated-files-from-go-test-coverage

v1v · 2022-04-28T13:15:01Z

@ph,
do you think we can move on with this PR as is? Then someone in your team can help with ignoring nay giving list of files from the code coverage report in a follow up?

jlind23 · 2022-04-28T14:02:20Z

@v1v sure move ahead, we will update it afterwards.

v1v · 2022-04-28T15:19:18Z

I only need some 👍 in the code review, so I can merge it :)

(cherry picked from commit 752c699)

(cherry picked from commit 752c699) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

…use-orka * 'main' of github.com:elastic/elastic-agent: (23 commits) [Automation] Update go release version to 1.17.10 (elastic#432) [Automation] Update elastic stack version to 8.3.0-4149272f for testing (elastic#435) [Automation] Update elastic stack version to 8.3.0-19aba912 for testing (elastic#430) Add extra k8s resources in clusterRole (elastic#424) [Automation] Update elastic stack version to 8.3.0-8ee1196f for testing (elastic#422) [Automation] Update elastic stack version to 8.3.0-53513548 for testing (elastic#421) Add tags option during enroll/install (elastic#336) validate kubernetes templates in .CI (elastic#417) add missing kube-api resources from managed agent manifest (elastic#381) Create snyk-scan.yml (elastic#397) [Automation] Update elastic stack version to 8.3.0-d380914f for testing (elastic#414) [Automation] Update elastic stack version to 8.3.0-5c1ff35f for testing (elastic#413) [Automation] Update elastic stack version to 8.3.0-6ba9f710 for testing (elastic#410) [Automation] Update elastic stack version to 8.3.0-a1c5cfff for testing (elastic#406) [Automation] Update elastic stack version to 8.3.0-7f585873 for testing (elastic#401) [Automation] Update elastic stack version to 8.3.0-0b6ea9f2 for testing (elastic#399) ci: enable coverage (elastic#377) Remove last dependencies on beats repo (elastic#387) Remove dependency on libbeat (elastic#344) [Automation] Update elastic stack version to 8.3.0-cb2ce38c for testing (elastic#383) ...

This is something that was introduced with elastic#377 and was meant to be used with Jenkins to show coverage report. Now with Jenkins been replace and BK + Sonar enabled on the repo, there is no need to create such artifacts anymore. Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

v1v added 2 commits April 26, 2022 13:52

ci: enable coverage

3b4aa43

ci: enable codecov in xml format

e52938f

mergify bot assigned v1v Apr 26, 2022

mergify bot added the backport-skip label Apr 26, 2022

fix dependencies

105332c

mdelapenya reviewed Apr 26, 2022

View reviewed changes

v1v added 7 commits April 26, 2022 15:55

fix location

bc063db

fix deps

2a51c32

fix location

d59fd66

mage: gocover-cobertura

f695f0e

use tools

ffc7357

deps

52bcf22

install required tool

5169f7c

v1v marked this pull request as ready for review April 26, 2022 18:45

v1v requested a review from a team as a code owner April 26, 2022 18:45

v1v requested review from a team, aleksmaus and narph and removed request for a team April 26, 2022 18:45

v1v commented Apr 26, 2022

View reviewed changes

jlind23 reviewed Apr 26, 2022

View reviewed changes

ci: enable cobertura on linux only

499e299

v1v added backport-v8.2.0 backport-v8.3.0 Automated backport with mergify labels Apr 27, 2022

mergify bot removed the backport-skip label Apr 27, 2022

kuisathaverat approved these changes Apr 28, 2022

View reviewed changes

cachedout approved these changes Apr 28, 2022

View reviewed changes

v1v merged commit 752c699 into elastic:main Apr 28, 2022

mergify bot pushed a commit that referenced this pull request Apr 28, 2022

ci: enable coverage (#377)

f3ba050

(cherry picked from commit 752c699)

mergify bot mentioned this pull request Apr 28, 2022

[8.2](backport #377) ci: enable coverage #389

Merged

v1v added a commit that referenced this pull request May 17, 2022

ci: enable coverage (#377) (#389)

037ffaf

(cherry picked from commit 752c699) Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>

Conversation

v1v commented Apr 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Questions

Tasks

Uh oh!

mergify bot commented Apr 26, 2022

Uh oh!

elasticmachine commented Apr 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

mdelapenya Apr 26, 2022

Choose a reason for hiding this comment

Uh oh!

mdelapenya Apr 26, 2022

Choose a reason for hiding this comment

Uh oh!

mdelapenya Apr 26, 2022

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Apr 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

v1v Apr 26, 2022

Choose a reason for hiding this comment

Uh oh!

jlind23 left a comment

Choose a reason for hiding this comment

Uh oh!

ph commented Apr 26, 2022

Uh oh!

v1v commented Apr 26, 2022

Uh oh!

jlind23 commented Apr 27, 2022

Uh oh!

v1v commented Apr 27, 2022

Uh oh!

ph commented Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

v1v commented Apr 28, 2022

Uh oh!

jlind23 commented Apr 28, 2022

Uh oh!

v1v commented Apr 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

v1v commented Apr 26, 2022 •

edited

Loading

elasticmachine commented Apr 26, 2022 •

edited

Loading

elasticmachine commented Apr 26, 2022 •

edited

Loading

ph commented Apr 27, 2022 •

edited

Loading