Skip to content

Use pytest-split to add some concurrency to integration test#1360

Merged
bharat-thotakura merged 19 commits into
mainfrom
bt/use-pytest-split
Mar 18, 2026
Merged

Use pytest-split to add some concurrency to integration test#1360
bharat-thotakura merged 19 commits into
mainfrom
bt/use-pytest-split

Conversation

@bharat-thotakura

@bharat-thotakura bharat-thotakura commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

Presently, the daily integration test workflow uses only 2 workers from pytest-xdist and generally can take a while to finish, waiting on some slower tests relative to others. Anticipating #1358, which will slow down the workflow even more, this PR adds pytest-split to break up the (current) total of 44 unit tests from the css and qss integration tests combined into 4 concurrent workflows, each using 2 workers. (Note: the tests are split up evenly across the 4 workflows (called 'groups'), but pytest-split can unevenly group time-consuming tests if given pytest timing info)

Additionally, on a more speculative & hopeful note, this PR also applies pytest-split on the three notebook checks in the regular CI to continue to mitigate against #1129 (with the idea being that each concurrent workflow should now only assign 1 notebook per worker (of course, subject to change if new notebooks are added)). In initial testing, this did not seem to offer much in time reduction due to the small number of notebooks being tested to begin with (& the worker overhead perhaps), but this approach might still be worth experimenting with.

Lastly, to confer any potential speed benefits, this PR also updates all relevant CI jobs to use Python 3.14 from the previous Python 3.13

Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
@bharat-thotakura

Copy link
Copy Markdown
Contributor Author

Example workflow with this PR

Comment thread .github/workflows/ci.yml Outdated
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install ./checks-superstaq ./general-superstaq[dev] ./qiskit-superstaq[dev] ./cirq-superstaq[dev]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getting outside the scope of this pr, but it'd probably be nice to also split up the css and qss integration tests (with each installing just the relevant packages)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added (see this workflow for an example)

Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>

@richrines1 richrines1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg % a couple nits

Comment thread .github/workflows/daily-integration-check.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
bharat-thotakura and others added 2 commits March 18, 2026 10:36
Co-authored-by: richrines1 <85512171+richrines1@users.noreply.github.com>
Co-authored-by: richrines1 <85512171+richrines1@users.noreply.github.com>
@bharat-thotakura bharat-thotakura merged commit e47976a into main Mar 18, 2026
24 checks passed
@bharat-thotakura bharat-thotakura deleted the bt/use-pytest-split branch March 18, 2026 17:58
bharat-thotakura added a commit that referenced this pull request Apr 8, 2026
#1360 added usage of
`pytest-split` to the CI's notebook checks in hopes of mitigating
against #1129.
However, this has been falsified by a [notebook check
flake](https://github.com/Infleqtion/client-superstaq/actions/runs/24104137963/job/70322973623)
occurring regardless. As such, the PR reverts some of the changes
introduced in #1360
to reduce redundant CI minute usage for apparent minimal benefit (at
this point at least)

Signed-off-by: Bharath <bharath.thotakura@infleqtion.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants