Skip to content

chore: Add script to detect which tests to run in CI#1563

Merged
HofmeisterAn merged 6 commits intodevelopfrom
feature/run-only-necessary-test-projects
Nov 2, 2025
Merged

chore: Add script to detect which tests to run in CI#1563
HofmeisterAn merged 6 commits intodevelopfrom
feature/run-only-necessary-test-projects

Conversation

@HofmeisterAn
Copy link
Collaborator

What does this PR do?

This PR adds a script that filters which test projects need to run based on the list of changed files.
The CI workflow gathers all available test projects and passes them to the script for filtering.

For the main and develop branches, all tests still run as usual.

@digital88 WDYT? It's quite aligned with your version. This version doesn't yet make use of file extensions, which we should probably include. I also like the comments and explanations in your version, which we should add as well.

Why is it important?

This should help reduce CI usage and overall build time, especially for pull requests.

Related issues

@HofmeisterAn HofmeisterAn added the chore A change that doesn't impact the existing functionality, e.g. internal refactorings or cleanups label Nov 1, 2025
@netlify
Copy link

netlify bot commented Nov 1, 2025

Deploy Preview for testcontainers-dotnet ready!

Name Link
🔨 Latest commit 480f294
🔍 Latest deploy log https://app.netlify.com/projects/testcontainers-dotnet/deploys/6907109f9e67480008d12158
😎 Deploy Preview https://deploy-preview-1563--testcontainers-dotnet.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link

coderabbitai bot commented Nov 1, 2025

Summary by CodeRabbit

  • New Features

    • Implemented intelligent test selection that runs only relevant tests based on changed files and branch context, improving CI/CD efficiency.
  • Bug Fixes

    • Fixed test reporting to gracefully handle scenarios with no test results without causing failures.
  • Chores

    • Optimized CI/CD pipeline workflow for streamlined execution.

Walkthrough

The changes implement selective test execution in CI pipelines by analyzing changed files and branch context. A new decision engine in Filter-TestProjects.ps1 filters which test projects run based on protected branches, global changes, and module-specific patterns. Collection scripts ensure safe JSON output, and workflow conditions prevent running CI jobs when no tests are collected.

Changes

Cohort / File(s) Summary
Test Project Filtering Logic
\.github/scripts/Filter-TestProjects\.ps1
Introduces two new functions: Should-RunTests (implements multi-rule decision logic for test selection based on branch protection, global changes, and module-specific patterns) and Filter-TestProjects (pipeline-friendly filtering with Begin/Process/End stages). Replaces minimal passthrough with structured, rule-based filtering driven by environment variables.
Test Collection Safety
\.github/scripts/Collect-TestProjects\.ps1
Replaces direct piping with intermediate variable assignment and adds null-coalescing fallback (?? "[]") to ensure safe JSON array output. Changes output structure to handle cases where filtering yields no results.
Workflow Configuration
\.github/workflows/cicd\.yml
Adds a new step to collect changed files using tj-actions/changed-files@v47, exposes files via ALL_CHANGED_FILES environment variable, and adds conditional for the ci job to run only when test-projects list is not empty.
Test Reporting Configuration
\.github/workflows/test-report\.yml
Adds fail-on-empty: false configuration to the Publish Test Report step to prevent workflow failure when no test results are available.

Sequence Diagram

sequenceDiagram
    actor Workflow as CI Workflow
    participant Collect as Collect-TestProjects.ps1
    participant Filter as Filter-TestProjects.ps1
    participant SRT as Should-RunTests
    participant Env as Environment<br/>(Branch, Changed Files)
    
    Workflow->>Collect: Invoke with test projects
    Collect->>Filter: Pipe test projects
    
    loop For each test project
        Filter->>SRT: Invoke Should-RunTests(ModuleName)
        SRT->>Env: Read branch & changed files
        alt Protected Branch
            SRT-->>Filter: Include test
        else Global Changes Detected
            SRT-->>Filter: Include test
        else Module-Specific Changes
            SRT-->>Filter: Include test
        else No Relevant Changes
            SRT-->>Filter: Skip test
        end
    end
    
    Filter->>Collect: Output filtered projects
    Collect->>Workflow: Return JSON array or []
    
    alt Non-empty test-projects
        Workflow->>Workflow: Run CI job
    else Empty test-projects
        Workflow->>Workflow: Skip CI job (conditional)
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–25 minutes

  • Key areas requiring attention:
    • Logic in Should-RunTests function: verify all decision rules are correct and cover intended scenarios (protected branches, global patterns, module categories)
    • Configuration arrays for protected branches, global changes, and module categorization: ensure they match actual repository structure and conventions
    • Integration between Collect-TestProjects.ps1, Filter-TestProjects.ps1, and workflow steps: validate the piping, variable passing, and environment variable usage
    • Workflow conditional logic: confirm that needs.collect-test-projects.outputs.test-projects != '[]' correctly prevents empty runs
    • Null-coalescing fallback in collection script: ensure it produces valid JSON in edge cases

Poem

🐰 Hoppity-hop through the CI flow,
With smart filters that help tests grow!
No more all-or-nothing runs today,
Just the changes' tests have their say! 🎯

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Out of Scope Changes Check ⚠️ Warning Consider extracting the test-report.yml modification into a separate PR focused on improving test report handling for edge cases, or provide justification for why this change is necessary as part of the test detection feature. If retained, add a comment in the PR description explaining how the fail-on-empty: false configuration relates to the test detection objectives.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (3 passed)
Check name Status Explanation
Title Check ✅ Passed The PR title "chore: Add script to detect which tests to run in CI" clearly and accurately summarizes the main change. It follows conventional commit format, is specific enough that a developer scanning commit history would understand what was added, and directly corresponds to the implementation of a test detection script. The title is neither vague nor misleading, and it properly emphasizes the primary objective of the changeset.
Linked Issues Check ✅ Passed The PR implementation directly addresses the objectives from issue #1417, which calls for a script/automation in the CI pipeline to detect which tests need to run based on changed files to reduce CI work. The new Filter-TestProjects.ps1 script implements this detection logic with rules for protected branches, global changes, and module-specific changes. The cicd.yml modifications expose changed files via environment variables and conditionally execute the CI pipeline. These changes also partially address issue #1136 by providing the mechanism to reduce CI workload and disk space pressure through selective test execution rather than running all tests on every change.
Description Check ✅ Passed The PR description addresses the two mandatory template sections: "What does this PR do?" explains that the PR adds a filtering script based on changed files with special handling for protected branches, and "Why is it important?" clearly states the goal of reducing CI usage and build time. The description also includes the recommended "Related issues" section with two issue references. While the optional "How to test this PR" and "Follow-ups" sections are absent, the author does provide forward-looking commentary about needed improvements (file extensions, additional comments). The description contains sufficient detail to understand the change scope and intent.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/run-only-necessary-test-projects

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4a72cb3 and 480f294.

📒 Files selected for processing (4)
  • .github/scripts/Collect-TestProjects.ps1 (1 hunks)
  • .github/scripts/Filter-TestProjects.ps1 (1 hunks)
  • .github/workflows/cicd.yml (1 hunks)
  • .github/workflows/test-report.yml (1 hunks)

@HofmeisterAn HofmeisterAn marked this pull request as ready for review November 2, 2025 09:32
@digital88
Copy link
Contributor

What does this PR do?

This PR adds a script that filters which test projects need to run based on the list of changed files. The CI workflow gathers all available test projects and passes them to the script for filtering.

For the main and develop branches, all tests still run as usual.

@digital88 WDYT? It's quite aligned with your version. This version doesn't yet make use of file extensions, which we should probably include. I also like the comments and explanations in your version, which we should add as well.

Why is it important?

This should help reduce CI usage and overall build time, especially for pull requests.

Related issues

I think file extensions are not necessary here, regex matching $GLOBAL_PATTERNS probably covers all cases. I used to check extensions in my version just to shorten $GLOBAL_PATTERNS a bit (mine has only "src/Testcontainers/").

@HofmeisterAn HofmeisterAn merged commit 593912e into develop Nov 2, 2025
140 checks passed
@HofmeisterAn HofmeisterAn deleted the feature/run-only-necessary-test-projects branch November 2, 2025 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore A change that doesn't impact the existing functionality, e.g. internal refactorings or cleanups

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[README]: Contributing new modules

2 participants