Skip to content

Show the code coverage in make test-binary output.#594

Merged
jnovy merged 1 commit intocontainers:mainfrom
jankaluza:coverage
Sep 16, 2025
Merged

Show the code coverage in make test-binary output.#594
jnovy merged 1 commit intocontainers:mainfrom
jankaluza:coverage

Conversation

@jankaluza
Copy link
Member

This commit changes the make test-binary to recompile the conmon with --coverage flag so it generates the .gcno and .gcda(these files are included in the.gitignore` now) while running the tests.

After that, the gcovr is executed to show the code coverage based on those files.

@packit-as-a-service
Copy link

Ephemeral COPR build failed. @containers/packit-build please check.

@jankaluza
Copy link
Member Author

Output is visible at the end of https://cirrus-ci.com/task/6362001716281344.

@jnovy jnovy self-requested a review September 10, 2025 12:40
@jnovy jnovy added the jira label Sep 10, 2025
@jnovy
Copy link
Collaborator

jnovy commented Sep 10, 2025

@jankaluza Can we use just normal gcov so that we don't need to add a new package dependency and maybe adding a separate test-coverage target instead?

Something like this?

diff --git a/Makefile b/Makefile
index 90ff8f0..72b25a0 100644
--- a/Makefile
+++ b/Makefile
@@ -79,6 +79,15 @@ test-binary: bin/conmon
 test: bin/conmon
        CONMON_BINARY="$(MAKEFILE_PATH)bin/conmon" test/run-tests.sh
 
+.PHONY: test-coverage
+test-coverage: clean
+       $(MAKE) bin/conmon DEBUGFLAG="--coverage"
+       CONMON_BINARY="$(MAKEFILE_PATH)bin/conmon" test/run-tests.sh
+       @echo "=== Coverage Summary ==="
+       @for file in src/*.c; do \
+               gcov $$file 2>/dev/null | grep -E "(^File|^Lines executed)" | paste - - | sed 's/File '\''src\/\([^.]*\)\.c'\''.*Lines executed:\([0-9.]*%\).*/\1.c: \2/'; \
+       done | grep -E "\.c: [0-9]" | sort
+
 bin:
        mkdir -p bin
 
@@ -90,7 +99,7 @@ docs:
 
 .PHONY: clean
 clean:
-       rm -rf bin/ src/*.o
+       rm -rf bin/ src/*.o src/*.gcda src/*.gcno *.gcov
        $(MAKE) -C docs clean
 
 .PHONY: install install.bin install.crio install.podman podman crio

@jankaluza
Copy link
Member Author

Separate test-coverage target is probably good idea.

For the plain gcov - it does not print the missing line numbers which is one of the reasons I wanted to add this test. I'm using that to find out untested parts of the code for which I should write a tests. I know this is not the only metric to use, but it gives a good overview.

The .gcov file-format is quite complex when it comes to uncovered lines. We could do some approximation of what gcovr is doing with:

	@echo "=== Missing coverage ==="
	@for file in *.gcov; do \
		awk -F':[ \t]*' '/#####|%%%%%/ { sub(/-.*/,"",$$2); print $$2 }' $$file | sort -nu | paste -sd, | awk -v f=$$file '{print f ": " $$0}'; \
	done | sort

But it does not handle some of the .gcov file-format features which are not trivial to handle without more complex code. I think gcovr is not that big to not justify its use. But I can also imagine using gcovr myself locally for my use-case and using just gcov in our tests for an overview.

@jankaluza jankaluza force-pushed the coverage branch 2 times, most recently from 648fb6f to 4dda1b6 Compare September 12, 2025 10:04
@jankaluza
Copy link
Member Author

@jnovy , there's a dedicated test-coverage target now and the code to fallback to gcov if gcovr is not found.

I believe the cri-o oldstable failure is not related? I will check it more to be sure.

This commit adds the `make test-coverage` to recompile the conmon
with `--coverage` flag so it generates the `.gcno and `.gcda` (these
files are included in the `.gitignore` now) while running the tests.

After that, the `gcovr` is executed to show the code coverage based on
those files.

If `gcovr` is not found, it fallbacks to `gcov`.

Signed-off-by: Jan Kaluza <jkaluza@redhat.com>
@jnovy
Copy link
Collaborator

jnovy commented Sep 16, 2025

LGTM now, thanks.

@jnovy jnovy merged commit 462e2b9 into containers:main Sep 16, 2025
32 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants