split up chrome tests#19209
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR splits Chrome system tests into smaller parallel groups to reduce CI/CD execution time. The monolithic chrome test suite is divided into 6 logical groups based on functionality: chrome_annotations, chrome_list, chrome_table, chrome_language, chrome_roleDescription, chrome_misc_aria, and chrome_misc.
Key changes:
- Tests are reorganized with tags for logical grouping
- CI/CD workflow updated to run each chrome test group in parallel
- License header updated to 2025 and modified to reference NVDA license terms
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
tests/system/robot/chromeTests.robot |
Reorganized tests into tagged groups; updated copyright year and license header |
.github/workflows/testAndPublish.yml |
Replaced single chrome test suite with 6 parallel chrome test groups |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
seanbudd
commented
Nov 13, 2025
SaschaCowley
approved these changes
Nov 14, 2025
5 tasks
seanbudd
added a commit
that referenced
this pull request
Nov 17, 2025
Follow up to #19209 Summary of the issue: We added a 10min time limit to cap and measure builds. This was just too low, as a recent build narrowly timed out. A similar cap for the create launcher step also is timing out: https://github.com/nvaccess/nvda/actions/runs/19415041964 Description of user facing changes: none Description of developer facing changes: builds shouldnt timeout unexpectedly Description of development approach: increase build timeout to 15min and launcher timeout to 30min
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to issue number:
Starts #18502
Summary of the issue:
Our system test bottle neck is chrome tests. There are too many chrome tests. They take up 20 min off time which could be parallelized down to 3-5min chunks
Description of user facing changes:
None
Description of developer facing changes:
Reduced PR times, better grouping logic when adding to chromeTests.robot.
Symbols tests are now the longest tests (6min). That means 14min from builds is shaved off from this PR.
Builds have gone from 40-45min to 30min on average.
Description of development approach:
Added tags to group chrome tests into sub groups.
This is to establish grounds for a potential refactor in the future.
Added timeout constraints to the 3 bottlenecks of the build: building NVDA source, building the launcher and system tests.
This is so we can monitor and constrain build time lengths.
With these constraints, we guarantee a 40min or faster build.
Testing strategy:
GitHub actions runners
Known issues with pull request:
This doesn't refactor the test files fully, just begins the proposal for logical groupings.
Further work is suggested to refactor chromeTests.py/chromeTests.robot
Code Review Checklist: