We currently upload the code coverage for every platform we run the test suite on but we have no way of telling which reported coverage is from which platform. It's not clear either what Codecov does when it receives multiple code coverage reports for the same commit. Does it save only the 1st one it receives? the last one? does it somehow save every of them? But then, which one gets displayed in PRs?

I've also noticed weird code coverage behaviors in the CI pipeline (e.g. #568) and I suspect some of it has to do with this quirky setup.

Moreover, CI might end up failing simply because of coverage differences between two platforms even if a PR has total coverage.

I suggest we choose a reference platform for code coverage and to stick to it.

How?

We should make coverage upload happen only for the reference platform in .github/workflows/test.yml.

Reference platform

We currently run tests on Linux and Windows platforms. Arguably, Linux platforms are more important than Windows platforms given most super-computers run Linux-based operating systems so I suggest we choose a Linux platform as the reference. We also run the tests on multiple versions of Python and I suggest somewhat arbitrarily to pick the minimum version of Python we support for the reference platform.

This would currently amount to picking ubuntu-latest with Python 3.10 as the reference platform.