[v2] Randomize test order for unit tests#3434
Merged
stephenfin merged 8 commits intogophercloud:v2from Jul 16, 2025
Merged
Conversation
Global tests objects are harmful, in so far as any attempt to modify them can result in failures further down the food chain. Don't use them and instead generate new objects on the fly. Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 610f545)
The fake service clients rely on the fake server being initialised, even if we don't actually use it. Add the missing calls, renaming some tests in the process. PS: You can reproduce this without the '-shuffle on' by simply deleting any of the test cases preceding those that we are modifying here and running the tests. Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 47b361b)
This is a mildly backwards-incompatible change, but since most people only use this API on the level of pagination.Pager, it ought not be a big deal in practice. Closes gophercloud#3222. (cherry picked from commit 97c0a32)
Ahead of some large-scale rework here. Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 1476dae)
All aliases were discovered with:
ag '[^\t]"github.com/gophercloud/gophercloud/v2/testhelper/client"'
They were then replaced with judicious use of sed and ag. Starting with
the most common alias, 'fake', first replace the alias:
sed -i 's/\<fake\>/client/' $(ag 'fake "github.com/gophercloud/gophercloud/v2/testhelper/client"' -l)
This was repeated for the other aliases, 'fakeclient' and 'thclient'. We
then dropped the now unnecessary aliasing:
sed -i 's/\<client\> "/"/' $(ag 'client "github.com/gophercloud/gophercloud/v2/testhelper/client"' -l)
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
(cherry picked from commit beeed5b)
Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 2f52485)
The meat of this changes is in 'testhelper/http_responses.go' and
'testhelper/client/fake.go'. The rest of the change is bulk generated
updates to handle those changes:
Create a "FakeServer" instance instead of configuring the global fake:
sed -i 's/th.SetupHTTP/fakeServer := th.SetupHTTP/g' $(ag 'th.SetupHTTP' -l)
Tear down this instance on test completion:
sed -i 's/defer th.TeardownHTTP/defer fakeServer.Teardown/g' $(ag 'defer th.TeardownHTTP' -l)
Update calls to the global 'Mux' instance with calls to the FakeServer
version:
sed -i 's/th.Mux.HandleFunc/fakeServer.Mux.HandleFunc/g' $(ag 'th.Mux.HandleFunc' -l)
Update calls to ServiceClient to include a 'FakeServer' parameter:
sed -i 's/client.ServiceClient()/client.ServiceClient(fakeServer)/' $(ag -w client.ServiceClient -l)
Plus alises for the same:
sed -i 's/fake.ServiceClient()/fake.ServiceClient(fakeServer)/' $(ag 'fake.ServiceClient' -l)
Update definitions and calls to various MockFoo and HandleFoo helpers -
plus lowercase, private variants of same - to include a 'FakeServer'
parameter:
sed -i 's/\(func Mock\w\+(t \*testing.T\)/\1, fakeServer th.FakeServer/' $(ag 'func Mock\w+\(t \*testing.T' -l)
sed -i 's/\(\tMock\w\+(t\)/\1, fakeServer/' $(ag '\tMock\w+\(t' -l)
sed -i 's/\(func mock\w\+(t \*testing.T\)/\1, fakeServer th.FakeServer/' $(ag 'func mock\w+\(t \*testing.T' -l)
sed -i 's/\(\tmock\w\+(t\)/\1, fakeServer/' $(ag '\tmock\w+\(t' -l)
sed -i 's/\(func Handle\w\+(t \*testing.T\)/\1, fakeServer th.FakeServer/' $(ag 'func Handle\w+\(t \*testing.T' -l)
sed -i 's/\(\tHandle\w\+(t\)/\1, fakeServer/' $(ag '\tHandle\w+\(t' -l)
sed -i 's/\(func handle\w\+(t \*testing.T\)/\1, fakeServer th.FakeServer/' $(ag 'func handle\w+\(t \*testing.T' -l)
sed -i 's/\(\thandle\w\+(t\)/\1, fakeServer/' $(ag '\thandle\w+\(t' -l)
Finally, there were some manual fixes to handle stuff that the regexes
missed.
Conflicts:
openstack/baremetal/v1/portgroups/testing/fixtures.go
openstack/baremetal/v1/portgroups/testing/requests_test.go
openstack/compute/v2/servers/testing/requests_test.go
openstack/db/v1/quotas/testing/fixtures_test.go
openstack/db/v1/quotas/testing/requests_test.go
openstack/dns/v2/zones/testing/requests_test.go
openstack/loadbalancer/v2/flavors/testing/fixtures.go
openstack/loadbalancer/v2/flavors/testing/requests_test.go
openstack/networking/v2/extensions/security/rules/testing/requests_test.go
Changes:
openstack/blockstorage/v3/manageablevolumes/testing/fixtures_test.go
openstack/blockstorage/v3/manageablevolumes/testing/requests_test.go
openstack/blockstorage/v3/volumes/testing/fixtures_test.go
openstack/blockstorage/v3/volumes/testing/requests_test.go
openstack/networking/v2/extensions/security/addressgroups/testing/requests_test.go
NOTE(stephenfin): Conflicts are generally due to tests or test files
that don't exist on the v2 branch. Changes are due to tests that have
been removed or moved on the main branch.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
(cherry picked from commit eed41cd)
I don't know why this isn't the default behavior, but relying on test order is a bad idea (TM) that we should do our best to avoid. Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 9251e99)
Contributor
Author
|
idk if we want to include fcf44f4 here. It's technically a breaking change but it's also a bugfix. Input welcome. |
Contributor
|
@stephenfin I don't see anything breaking there. |
Contributor
Author
From the |
Contributor
|
I've never seen that someone used direct access to |
Contributor
Author
In that case, I think we're safe to merge this. |
kayrus
approved these changes
Jul 15, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is manual backport of #3108, with trivial conflicts on the penultimate patch addressed.
This is not critical, but backporting it will ease future backports from the
mainbranch.