Skip to content

Backend snapshot tests: separate snapshot for each major version#1504

Merged
NachoSoto merged 5 commits into
mainfrom
snapshot-testing-os-versions
Apr 14, 2022
Merged

Backend snapshot tests: separate snapshot for each major version#1504
NachoSoto merged 5 commits into
mainfrom
snapshot-testing-os-versions

Conversation

@NachoSoto

@NachoSoto NachoSoto commented Apr 13, 2022

Copy link
Copy Markdown
Contributor

Fixes CF-547.

I recommend reviewing this ignoring JSON files:https://github.com/RevenueCat/purchases-ios/pull/1504/files?file-filters%5B%5D=.pbxproj&file-filters%5B%5D=.swift&show-deleted-files=true&show-viewed-files=true

This also combines the duplicate copies of some tests for each version, as only 1 is needed now. assertSnapshot encodes the version in the filename.
Combined:

  • BackendPostRecepitDataTests
  • BackendSubscriberAttributesTests

No more unnecessary skipped tests.

  • Before: Number of tests skipped | 112
  • Now: Number of tests skipped | 58

Example:

Screen Shot 2022-04-13 at 16 07 15

⚠️ Note:

Because iOS 12.x simulators currently crash on M1 Macs, I haven't generated those snapshots yet.

@NachoSoto NachoSoto requested a review from a team April 13, 2022 20:06
Comment on lines 33 to 52

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the core of this PR. Thanks @taquitos for the idea.

@taquitos taquitos left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome stuff, one possible change I see some files prefixed like iOSiOS15

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🕺

@NachoSoto

Copy link
Copy Markdown
Contributor Author

Oops good catch! I forgot we have those separate tests as well.

@NachoSoto

Copy link
Copy Markdown
Contributor Author

Combined BackendSubscriberAttributesTests as well and updated snapshots.

@NachoSoto NachoSoto requested a review from taquitos April 13, 2022 21:09

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BackendPostReceiptDataTest -> BackendPostReceiptDataTests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops thanks

@taquitos taquitos left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The smallest comment update, but then 🚢

@NachoSoto

Copy link
Copy Markdown
Contributor Author

I'll wait to merge this until somebody can generate the iOS 12 snapshots for me 🙏🏻

@taquitos

Copy link
Copy Markdown
Contributor

I'll wait to merge this until somebody can generate the iOS 12 snapshots for me 🙏🏻

updating my Intel mac 🙃

Fixes [CF-547].

This also removes the duplicate `BackendPostRecepitDataTests` for each version, as only 1 is needed now. `assertSnapshot` encodes the version in the filename.
@NachoSoto NachoSoto force-pushed the snapshot-testing-os-versions branch from 5fff36d to 8d8cec1 Compare April 13, 2022 21:31
@taquitos

Copy link
Copy Markdown
Contributor

@NachoSoto this is ready when you are 😄

@NachoSoto

Copy link
Copy Markdown
Contributor Author

Awesome thanks! Just pushed the comment fix, I'll merge this and make sure they're passing in main.

@NachoSoto NachoSoto merged commit d75d580 into main Apr 14, 2022
@NachoSoto NachoSoto deleted the snapshot-testing-os-versions branch April 14, 2022 12:31
NachoSoto added a commit that referenced this pull request May 10, 2022
…utEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```
NachoSoto added a commit that referenced this pull request May 20, 2022
…utEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```
NachoSoto added a commit that referenced this pull request May 25, 2022
…utEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```
NachoSoto added a commit that referenced this pull request Jun 9, 2022
…utEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```
NachoSoto added a commit that referenced this pull request Jun 9, 2022
…utEscapingSlashes` if available (#1560)

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```
joshdholtz pushed a commit that referenced this pull request May 12, 2023
Fixes https://app.circleci.com/pipelines/github/RevenueCat/purchases-ios/6737/workflows/b945f5f6-6f31-4716-b1ac-6e16e3848c7e/jobs/26321

For some reason, on iOS 12.x only, this:
```swift
JSONDecoder.default.decode(jsonData: "null".data(using: .utf8)!)
```

Fails:
```
dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "JSON text did not start with array or object and option to allow fragments not set." UserInfo={NSDebugDescription=JSON text did not start with array or object and option to allow fragments not set.})))
```

We don't rely on decoding fragments, so I just updated the test to check `.null` within a dictionary instead.

`Snapshotting.formattedData`: use `JSONEncoder.OutputFormatting.withoutEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```

Update CircleCI config to use pipeline parameters to set generate snapshots env

Run all in one serially

Split OS version into their own jobs

Create PR from all os version jobs

This is the one

Added requestby user in body

Fix for pr actually creating

Reordered some things

Remove tvos from generated jobs

Raw output when generating snapshots

Don't retry tests when generating snapshots

Maybe this will make the PRs

Maybe this

This is it
joshdholtz pushed a commit that referenced this pull request May 12, 2023
Fixes https://app.circleci.com/pipelines/github/RevenueCat/purchases-ios/6737/workflows/b945f5f6-6f31-4716-b1ac-6e16e3848c7e/jobs/26321

For some reason, on iOS 12.x only, this:
```swift
JSONDecoder.default.decode(jsonData: "null".data(using: .utf8)!)
```

Fails:
```
dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "JSON text did not start with array or object and option to allow fragments not set." UserInfo={NSDebugDescription=JSON text did not start with array or object and option to allow fragments not set.})))
```

We don't rely on decoding fragments, so I just updated the test to check `.null` within a dictionary instead.

`Snapshotting.formattedData`: use `JSONEncoder.OutputFormatting.withoutEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```

Update CircleCI config to use pipeline parameters to set generate snapshots env

Run all in one serially

Split OS version into their own jobs

Create PR from all os version jobs

This is the one

Added requestby user in body

Fix for pr actually creating

Reordered some things

Remove tvos from generated jobs

Raw output when generating snapshots

Don't retry tests when generating snapshots

Maybe this will make the PRs

Maybe this

This is it
joshdholtz pushed a commit that referenced this pull request May 12, 2023
Fixes https://app.circleci.com/pipelines/github/RevenueCat/purchases-ios/6737/workflows/b945f5f6-6f31-4716-b1ac-6e16e3848c7e/jobs/26321

For some reason, on iOS 12.x only, this:
```swift
JSONDecoder.default.decode(jsonData: "null".data(using: .utf8)!)
```

Fails:
```
dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "The given data was not valid JSON.", underlyingError: Optional(Error Domain=NSCocoaErrorDomain Code=3840 "JSON text did not start with array or object and option to allow fragments not set." UserInfo={NSDebugDescription=JSON text did not start with array or object and option to allow fragments not set.})))
```

We don't rely on decoding fragments, so I just updated the test to check `.null` within a dictionary instead.

`Snapshotting.formattedData`: use `JSONEncoder.OutputFormatting.withoutEscapingSlashes` if available

This was removed after a [PR review comment](#1366 (comment))
because it would have lead to different snapshots in iOS 12.

However, since #1504 we have separate snapshots for each iOS version, so this cleans up the output when possible.

```diff
-    "url" : "https:\/\/api.revenuecat.com\/v1\/subscribers\/user\/alias"
+    "url" : "https://api.revenuecat.com/v1/subscribers/user/alias"
```

Update CircleCI config to use pipeline parameters to set generate snapshots env

Run all in one serially

Split OS version into their own jobs

Create PR from all os version jobs

This is the one

Added requestby user in body

Fix for pr actually creating

Reordered some things

Remove tvos from generated jobs

Raw output when generating snapshots

Don't retry tests when generating snapshots

Maybe this will make the PRs

Maybe this

This is it
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants