Skip to content

feat: Add test coverage reporting. Closes #701#712

Merged
mlodic merged 9 commits intointelowlproject:developfrom
opbot-xd:feat/test-coverage-reporting-701
Jan 21, 2026
Merged

feat: Add test coverage reporting. Closes #701#712
mlodic merged 9 commits intointelowlproject:developfrom
opbot-xd:feat/test-coverage-reporting-701

Conversation

@opbot-xd
Copy link
Copy Markdown
Contributor

Description

This PR implements comprehensive test coverage reporting for GreedyBear, enabling developers to track code coverage both locally and in CI.

Summary of changes:

  • Added .coveragerc configuration file to define coverage settings
  • Enhanced CI workflow to generate coverage reports and upload artifacts
  • Auto-install coverage in local Docker development environment
  • Added requirements/test.txt for non-Docker environments
  • Updated .gitignore to exclude coverage artifacts

Related issues

Closes #701

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read and understood the rules about how to Contribute to this project
  • The pull request is for the branch develop
  • I have added documentation of the new features.
  • Linter (Ruff) gave 0 errors
  • All the tests (old ones) gave 0 errors
  • If changes were made to an existing model/serializer/view, the docs were updated and regenerated (N/A - no model/serializer/view changes)
  • If the GUI has been modified (N/A - no GUI changes)

- Add .coveragerc configuration file
  - Define source paths (greedybear/)
  - Exclude migrations, tests, and generated files
  - Configure report formatting (show_missing, precision)
  - Set output formats for HTML and XML reports

- Update CI workflow to generate coverage artifacts
  - Generate XML coverage report for download
  - Upload coverage reports as GitHub Actions artifacts
  - Add coverage summary to job summary for quick visibility
  - Set 30-day retention for coverage artifacts

- Update .gitignore to exclude coverage artifacts
  - htmlcov/, .coverage, coverage.xml, etc.

This implements Phase 1 (Local Coverage Setup) and Phase 2 (CI Integration)
as discussed in issue intelowlproject#701. Coverage is already enabled in the CI workflow
and will now generate detailed reports.

Local usage:
  coverage run --source=greedybear manage.py test tests
  coverage report  # Terminal output
  coverage html    # Browser-viewable report in htmlcov/

Related: intelowlproject#701
@opbot-xd
Copy link
Copy Markdown
Contributor Author

Implementation Complete! 🎉

I've implemented Phase 1 (Local Coverage Setup) and Phase 2 (CI Integration) as discussed in #701.

How to Use

Local Development (Docker):

# Coverage is automatically installed when you start containers
docker compose up -d

# Run tests with coverage
docker compose exec uwsgi python3 -m coverage run --source=greedybear manage.py test tests

# View coverage report in terminal
docker compose exec uwsgi python3 -m coverage report -m

# Generate HTML report (browser-viewable)
docker compose exec uwsgi python3 -m coverage html
# Then open htmlcov/index.html in your browser

# Generate XML report
docker compose exec uwsgi python3 -m coverage xml

Non-Docker Environments:

pip install -r requirements/test.txt
coverage run --source=greedybear manage.py test tests
coverage report -m

CI (Automatic):
When you create a PR:

  1. Tests run with coverage automatically
  2. Coverage summary appears in the Actions tab (job summary)
  3. Download coverage artifacts from the workflow run (available for 30 days)

@opbot-xd
Copy link
Copy Markdown
Contributor Author

Hi @mlodic @regulartim

  1. Is this approach acceptable?
  2. Any other improvements you'd like to see before merging?

Would be soon updating the docs repo once this is merged.

@opbot-xd opbot-xd marked this pull request as ready for review January 16, 2026 15:09
Copilot AI review requested due to automatic review settings January 16, 2026 15:09
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements comprehensive test coverage reporting for GreedyBear, enabling developers to track code coverage both locally and in CI environments. The changes establish a foundation for monitoring test quality and identifying untested code paths.

Changes:

  • Added .coveragerc configuration file with coverage settings for source tracking, exclusions, and report formatting
  • Enhanced CI workflow to generate coverage XML reports and upload them as artifacts with 30-day retention
  • Auto-installed coverage package in local Docker development environment via modified entrypoint
  • Created requirements/test.txt for non-Docker environments

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.coveragerc Configures coverage.py with source paths, omissions, report formatting, and exclusion patterns
.github/workflows/_python.yml Adds coverage XML generation and artifact upload steps to CI pipeline
docker/local.override.yml Modifies uwsgi service entrypoint to install coverage package before starting
requirements/test.txt Defines test dependencies for non-Docker installations
.gitignore Excludes coverage artifacts from version control

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Add coverage auto-installation to local.override.yml
  - Coverage is now automatically installed when running docker compose
  - Developers no longer need to manually 'pip install coverage'

- Add requirements/test.txt for non-Docker environments
  - Provides easy installation: pip install -r requirements/test.txt
  - Documents test dependencies

This implements Option 4 as approved by @mlodic.

Related: intelowlproject#701
- DJANGO_TEST_SERVER=True
- DJANGO_WATCHMAN_TIMEOUT=20
entrypoint: >
sh -c "pip install --no-cache-dir coverage>=7.6.10 &&
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem about this is that we are not able to track updates of coverage package via dependabot. This is necessary from a maintainer point of view

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! You're absolutely right - we need coverage tracked by Dependabot.

The Problem

Currently, coverage is installed at runtime via local.override.yml, which means:

  • Dependabot can't track it
  • Version updates won't be automated

Proposed Solutions

I can implement either of these approaches:

Option A: Add to requirements/project-requirements.txt

+ # Test coverage
+ coverage>=7.3.2
  • Dependabot tracks it automatically
  • Simplest implementation
  • Coverage available everywhere (production, dev, CI)
  • Minimal overhead (~240KB)
  • Adds test dependency to production image

Option B: Create requirements/dev-requirements.txt

# Development requirements
coverage>=7.3.2

Then update Dockerfile to conditionally install dev requirements:

ARG INSTALL_DEV=false
RUN if [ "$INSTALL_DEV" = "true" ]; then \
      pip install -r requirements/dev-requirements.txt; \
    fi
  • Dependabot tracks it automatically
  • Clean separation (dev vs production)
  • Production image stays lean
  • More complex implementation
  • Requires Dockerfile changes

# Test requirements
# Automatically installed in local Docker setup via local.override.yml
# For manual installation: pip install -r requirements/test.txt
coverage>=7.6.10
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

@opbot-xd opbot-xd Jan 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at .github/dependabot.yml (lines 4-9):

- package-ecosystem: "pip"
  directory: "/requirements"

So requirements/test.txt should be tracked automatically by the existing Dependabot config.
However, I see your concern: the current implementation installs coverage at runtime via local.override.yml:

entrypoint: >
  sh -c "pip install --no-cache-dir coverage>=7.3.2 &&
         exec /opt/deploy/greedybear/docker/entrypoint_uwsgi.sh"

@@ -0,0 +1,33 @@
[run]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure that this file is actually taken into account in the github action?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! The .coveragerc file is used by the GitHub Actions workflow.

Evidence

In .github/workflows/_python.yml:

Line 85-89 - Config path parameter:

coverage_config_path:
  description: Path to the coverage.py config file
  type: string
  required: false
  default: .coveragerc

Line 513 - Coverage command uses the config:

CMD="coverage run --rcfile=${{ inputs.coverage_config_path }}"

Since pull_request_automation.yml doesn't override coverage_config_path, it uses the default value .coveragerc.

How it works

  1. PR triggers pull_request_automation.yml
  2. Calls _python.yml with use_coverage: true
  3. _python.yml uses default coverage_config_path: .coveragerc
  4. Tests run with: coverage run --rcfile=.coveragerc manage.py test tests
  5. Coverage respects our config (excludes migrations, tests, etc.)

@opbot-xd opbot-xd force-pushed the feat/test-coverage-reporting-701 branch from 4ef3a30 to c0a406c Compare January 16, 2026 15:14
Copilot AI review requested due to automatic review settings January 16, 2026 18:14
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@opbot-xd opbot-xd force-pushed the feat/test-coverage-reporting-701 branch from 7989aaf to c7fc345 Compare January 16, 2026 18:19
- Fixed if __name__ == __main__ pattern
- Changed from dots (.__main__.:) to proper quotes ("__main__":)
- Copilot suggestion applied

Co-authored-by: GitHub Copilot
Copilot AI review requested due to automatic review settings January 16, 2026 18:21
@opbot-xd opbot-xd force-pushed the feat/test-coverage-reporting-701 branch from c7fc345 to f69ea92 Compare January 16, 2026 18:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 5 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Removed .coverage file from artifact upload
- Prevents overwrites between different Python versions in matrix
- XML report is sufficient for coverage analysis
- Binary .coverage files are version-specific

Co-authored-by: GitHub Copilot
@opbot-xd
Copy link
Copy Markdown
Contributor Author

Hi @mlodic
Thanks for the feedback! You're absolutely right - we need coverage tracked by Dependabot.

The Problem

Currently, coverage is installed at runtime via local.override.yml, which means:

  • Dependabot can't track it
  • Version updates won't be automated

Proposed Solutions

I can implement either of these approaches:

Option A: Add to requirements/project-requirements.txt

+ # Test coverage
+ coverage>=7.3.2
  • Dependabot tracks it automatically
  • Simplest implementation
  • Coverage available everywhere (production, dev, CI)
  • Minimal overhead (~240KB)
  • Adds test dependency to production image

Option B: Create requirements/dev-requirements.txt

# Development requirements
coverage>=7.3.2

Then update Dockerfile to conditionally install dev requirements:

ARG INSTALL_DEV=false
RUN if [ "$INSTALL_DEV" = "true" ]; then \
      pip install -r requirements/dev-requirements.txt; \
    fi
  • Dependabot tracks it automatically
  • Clean separation (dev vs production)
  • Production image stays lean
  • More complex implementation
  • Requires Dockerfile changes

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jan 19, 2026

Fine with the second option from my perspective

Implements Option B as approved by @mlodic:

- Create requirements/dev-requirements.txt with coverage>=7.3.2
- Add INSTALL_DEV build argument to Dockerfile
- Update local.override.yml to use INSTALL_DEV=true
- Remove requirements/test.txt (replaced by dev-requirements.txt)

Benefits:
- Dependabot tracks dev dependencies automatically
- Clean separation between dev and production
- Production image stays lean (no coverage)
- Dev dependencies installed at build time, not runtime

Usage:
  docker compose up --build  # Local dev (includes coverage)
  docker build .             # Production (no dev deps)

Related: intelowlproject#701
Copilot AI review requested due to automatic review settings January 19, 2026 23:19
@opbot-xd opbot-xd force-pushed the feat/test-coverage-reporting-701 branch from c20b6ad to 9fb32f9 Compare January 19, 2026 23:19
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Artifact name 'coverage-report-3.13' becomes 'coverage-report-3-13'
- Prevents GitHub Actions naming conflicts with periods

Co-authored-by: GitHub Copilot
@opbot-xd
Copy link
Copy Markdown
Contributor Author

Hi @mlodic the implementation is complete now.
Please review it when you get time.

Re: Is .coveragerc used in GitHub Actions?

Yes! In _python.yml:

  • Line 89: coverage_config_path defaults to .coveragerc
  • Line 513: coverage run --rcfile=${{ inputs.coverage_config_path }}

Dependabot Tracking

Existing config in .github/dependabot.yml tracks /requirements directory. Since dev-requirements.txt is there, coverage updates are auto-tracked.

Usage (Docker)

docker compose exec uwsgi coverage run --source=greedybear manage.py test tests
docker compose exec uwsgi coverage report -m

@opbot-xd opbot-xd requested a review from mlodic January 19, 2026 23:27
Copilot AI review requested due to automatic review settings January 20, 2026 00:06
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@opbot-xd
Copy link
Copy Markdown
Contributor Author

opbot-xd commented Jan 20, 2026

image

Fix for CI failure

I got a notification that CI failed with:

Invalid workflow file: Unrecognized function: 'replace'

The previous commit used Copilot's suggestion replace(matrix.python_version, '.', '-') for the artifact name, but GitHub Actions doesn't have a replace function.

Fixed by using coverage-report-py${{ matrix.python_version }} instead. Periods are actually allowed in artifact names.

@opbot-xd
Copy link
Copy Markdown
Contributor Author

Hi @mlodic @regulartim can you run the CI / backend-tests, I just wanted to see whether the test coverage is working or not?
Thanks

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jan 20, 2026

the CI requires that at least a random change is pushed in the branch. You can just add a random log.debug() anywhere in the python code and that would do the trick. Then we can delete it before mergin the PR

@opbot-xd
Copy link
Copy Markdown
Contributor Author

@mlodic I beleive CI Test was successful
The coverage reporting feature is working. You can download the coverage XML report from:

coverage-report-py3.13

  • Tests ran with coverage enabled
  • coverage.xml generated successfully (4160 bytes)
  • Artifact uploaded with 30-day retention

Note

The last commit added a temporary debug log to trigger CI. Once you confirm everything looks good, I'll push a revert commit to remove it before merging.

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jan 21, 2026

it works, you can see the results here: https://github.com/intelowlproject/GreedyBear/actions/runs/21190092088

Copilot AI review requested due to automatic review settings January 21, 2026 12:59
@opbot-xd
Copy link
Copy Markdown
Contributor Author

Thanks @mlodic for testing
The temporary debug log has been reverted.
I think the PR is ready to merge or something is still left?

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mlodic mlodic merged commit 70ef310 into intelowlproject:develop Jan 21, 2026
9 of 10 checks passed
@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jan 21, 2026

good job, thanks!

@opbot-xd
Copy link
Copy Markdown
Contributor Author

Documentation Question

Hi @mlodic @regulartim,

Since test coverage is primarily a maintainer concern (contributors may not need to know about it), should I still add it to the contributor's section docs repo? Or is this feature internal-only and doesn't require public documentation?

Let me know. Thanks!

@mlodic
Copy link
Copy Markdown
Member

mlodic commented Jan 22, 2026

To me it is not strictly necessary, thanks for asking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants