Skip to content

Improve some of the flaky tests #954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
slang25 merged 8 commits into from
Jun 24, 2024
Merged

Improve some of the flaky tests #954

slang25 merged 8 commits into from
Jun 24, 2024

Conversation

@slang25
Copy link
Member

@slang25 slang25 commented Jun 23, 2024 •
edited
Loading

  • Use TaskCompletionSource to avoid task scheduler and waits where appropriate, these are unreliable on slow hardware (AppVeyor).
  • Standardise some of the wait durations, and have them be more lenient. The eventual solution to this would be TimeProvider, but that's a more substantial change.
  • Format HashCodes printing to a minimum of 6 digits (we'd occasionally get 4 digit hashcodes which was breaking the tests)
  • Improve HashCode scrubbing to handle all scenarios

I've tried to be careful about not changing the tests to act on delegates that are intended to not immediately complete, I think I've got the balance right here.

@slang25
Copy link
Member Author

slang25 commented Jun 23, 2024 •
edited
Loading

I'm going to run this 5 times, if these test continue to pass then I'll take it out of draft 🤞

  1. ✅ https://ci.appveyor.com/project/StuartLang/shouldly/builds/50075470
  2. ✅ https://ci.appveyor.com/project/StuartLang/shouldly/builds/50075472
  3. ✅ https://ci.appveyor.com/project/StuartLang/shouldly/builds/50075496
  4. ❌ https://ci.appveyor.com/project/StuartLang/shouldly/builds/50075517

@slang25
Copy link
Member Author

slang25 commented Jun 23, 2024

Ah, there are more tests of this style, will go after those too

@slang25 slang25 changed the title Simplify some of the flaky tests Improve some of the flaky tests Jun 23, 2024
@slang25 slang25 marked this pull request as ready for review June 23, 2024 15:16
@slang25
Copy link
Member Author

slang25 commented Jun 23, 2024 •
edited
Loading

Ok, now we are good I think:

  1. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075793
  2. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075796
  3. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075859
  4. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075908
  5. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075924
  6. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075937
  7. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075950
  8. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075980
  9. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50075997
  10. ✅ https://ci.appveyor.com/project/shouldly/shouldly/builds/50076025

@slang25 slang25 requested a review from SimonCropp June 23, 2024 15:40
@slang25
Copy link
Member Author

slang25 commented Jun 24, 2024

@SimonCropp I'll merge this unless you have any objection

@SimonCropp
Copy link
Contributor

SimonCropp commented Jun 24, 2024

LGTM

@slang25 slang25 merged commit 376f6fc into master Jun 24, 2024
@slang25 slang25 deleted the flaky-tests branch June 24, 2024 11:34
This was referenced Aug 4, 2025
Open
Closed
This was referenced Aug 23, 2025
Closed
Merged
This was referenced Sep 1, 2025
Open
Closed
Closed
Closed
Closed
This was referenced Sep 10, 2025
Closed
Closed
This was referenced Dec 9, 2025
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Merged
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Merged
Closed
This was referenced Dec 22, 2025
Closed
Closed
This was referenced Jan 5, 2026
Open
Closed
Merged
Merged
Merged
@dependabot dependabot bot mentioned this pull request Jan 12, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SimonCropp SimonCropp SimonCropp approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@slang25 @SimonCropp