Add support for tests executed repeatedly#585
Conversation
422288f to
44b3736
Compare
|
I have two questions regarding this
|
aad695f to
cae4380
Compare
Yes. TestObserver is called after test case is executed so the feature of counting multiple assertSnapshot calls works as expected.
I've never used Quick or Nimble. Would you be so kind to check with my forked version if it works with your project? I've already checked with our where we have planty of different scenarios but definitely would be cool to have confirmation also from someone else. |
|
@krzysztofpawski I tried it out and it still produces many snapshots. The way |
|
Maybe it will make sense to make it unique based on the test line. The only thing that would break it is if someone adds two assertions on the same line |
cae4380 to
922eee3
Compare
|
@tahirmt could you check it one more time? I found one more case when it was producing too many reference files in our repo. |
|
@krzysztofpawski I created a demo repo to test this using your fork https://github.com/tahirmt/test-toeventually-behavior-snapshots. |
|
@tahirmt I'm afraid it can't be fixed in |
That's what I was thinking too. I was thinking of many alternatives but I can't think of a better solution for now. Thanks for looking into it though. |
|
@stephencelis any possibility to review this PR? |
This includes [this fix](pointfreeco/swift-snapshot-testing#585) which should solve some of the flaky test failures we've been seeing lately, which made test retries also fail. We couldn't update to 1.10.0 because it dropped support for iOS 12.0, but that was fixed in pointfreeco/swift-snapshot-testing#698
This includes [this fix](pointfreeco/swift-snapshot-testing#585) which should solve some of the flaky test failures we've been seeing lately, which made test retries also fail. We couldn't update to 1.10.0 because it dropped support for iOS 12.0, but that was fixed in pointfreeco/swift-snapshot-testing#698 This also reverts #2426, since that's no longer required now that retries should work properly.
Motivation
Xcode 13introduced new parameters toxcodebuildthat allow to run tests repeatedly (wwdc talk).Unfortunately
SnapshotTestingdoesn't handle such situation in a correct way.Proposal
We can use
XCTestObservationin order to clean counter map between different test cases.Resolved issues
Closes #577