Integration Tests: another flaky failure#3165
Merged
Merged
Conversation
See https://app.circleci.com/pipelines/github/RevenueCat/purchases-ios/14903/workflows/a607f078-553a-44c0-8873-16abc12d3e3a/jobs/119243 This test (`testCallToGetCustomerInfoWithPendingTransactionsPostsReceiptOnlyOnce`) has a race condition. It currently does: - `purchaseMonthlyProduct` - `forceRenewalOfSubscription` However, with a slow CPU, it's possible that StoreKit will renew automatically, leading to more unfinished transactions. This modifies the test to be a bit more lenient, while still complying with the spirit of the test: `testCallToGetCustomerInfoWithPendingTransactionsPostsReceiptOnlyOnce`.
NachoSoto
commented
Sep 7, 2023
| @available(iOS 13.0, macOS 10.15, tvOS 13.0, watchOS 6.2, *) | ||
| func asyncWait( | ||
| description: String? = nil, | ||
| func asyncWait<T>( |
Contributor
Author
There was a problem hiding this comment.
I've added this new overload that allows printing the last value in the description.
This allows us to see exactly what the failure was, instead of just:
failed - Expected 2 unfinished transactions
expected to be true, got <false>
"Expected true, got false" wasn't very helpful.
NachoSoto
commented
Sep 7, 2023
| line: line, | ||
| foundCorrectValue | ||
| ).to(beTrue(), description: description) | ||
| ).to(beTrue(), description: description(lastValue)) |
Contributor
Author
There was a problem hiding this comment.
Now the description can be derived from the last value.
NachoSoto
commented
Sep 7, 2023
| ) | ||
|
|
||
| try await self.waitUntilUnfinishedTransactions(2) | ||
| try await self.waitUntilUnfinishedTransactions { $0 >= 2 } |
Contributor
Author
There was a problem hiding this comment.
This is the core of the change, from == 2 to >= 2.
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## main #3165 +/- ##
==========================================
- Coverage 86.64% 86.57% -0.07%
==========================================
Files 220 220
Lines 15742 15742
==========================================
- Hits 13639 13628 -11
- Misses 2103 2114 +11 ☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here. |
tonidero
approved these changes
Sep 8, 2023
This was referenced Sep 13, 2023
NachoSoto
pushed a commit
that referenced
this pull request
Sep 14, 2023
**This is an automatic release.** ### Bugfixes * Fix runtime crash in SK2TransactionListener in iOS < 15 (#3206) via Toni Rico (@tonidero) ### Performance Improvements * `OperationDispatcher`: add support for "long" delays (#3168) via NachoSoto (@NachoSoto) ### Other Changes * `Integration Tests`: add tests for ghost transfer behavior (#3135) via NachoSoto (@NachoSoto) * `Xcode`: removed `purchases-ios` SPM reference (#3166) via NachoSoto (@NachoSoto) * `Integration Tests`: another flaky failure (#3165) via NachoSoto (@NachoSoto) * `Integration Tests`: fix flaky test failure due to leftover transaction (#3167) via NachoSoto (@NachoSoto) * `Xcode 13`: removed last `Swift 5.7` checks (#3161) via NachoSoto (@NachoSoto) * `Integration Tests`: improve flaky tests (#3163) via NachoSoto (@NachoSoto) * `Codable`: improved decoding errors (#3153) via NachoSoto (@NachoSoto) * Refactor: extract `HealthOperation` (#3154) via NachoSoto (@NachoSoto) * `Xcode 13`: remove conditional code (#3147) via NachoSoto (@NachoSoto) * `CircleCI`: change all jobs to use `Xcode 14.x` and replace `xcode-install` with `xcodes` (#2421) via NachoSoto (@NachoSoto)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See https://app.circleci.com/pipelines/github/RevenueCat/purchases-ios/14903/workflows/a607f078-553a-44c0-8873-16abc12d3e3a/jobs/119243
This test (
testCallToGetCustomerInfoWithPendingTransactionsPostsReceiptOnlyOnce) has a race condition. It currently does:purchaseMonthlyProductforceRenewalOfSubscriptionHowever, with a slow CPU, it's possible that StoreKit will renew automatically, leading to more unfinished transactions.
This modifies the test to be a bit more lenient, while still complying with the spirit of the test:
testCallToGetCustomerInfoWithPendingTransactionsPostsReceiptOnlyOnce.