Add lane for running iOS v3 load shedder integration tests by aboedo · Pull Request #2388 · RevenueCat/purchases-ios

aboedo · 2023-03-30T13:52:29Z

This adds a fastlane lane to execute load shedder integration tests.
In a follow-up PR I'll call this lane automatically from CircleCI, on a schedule.

It won't be a part of regular integration tests, since this logic is tied to v3 and static. So only the load shedder itself is getting tested here.

aboedo · 2023-03-30T13:53:16Z

  end

+  desc "Run LoadShedder tests"
+  lane :v3_loadshedder_integration_tests do |options|


works on my machine ™️

Should we also have a job for v4 to run them on a schedule? We can reuse the existing jobs though, so it's probably ok?

we're already reusing the job for general backend integration tests.
I thought about separating them, but in practice it's certainly not a bad idea to also be doing all backend integration tests on v4 on a daily basis

codecov · 2023-03-30T15:26:10Z

Codecov Report

Merging #2388 (fffb3b9) into main (049fd9e) will increase coverage by 0.09%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #2388      +/-   ##
==========================================
+ Coverage   86.69%   86.78%   +0.09%     
==========================================
  Files         198      198              
  Lines       13214    13214              
==========================================
+ Hits        11456    11468      +12     
+ Misses       1758     1746      -12

see 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

tonidero · 2023-03-30T15:34:53Z

      paths_of_files_to_update: [
-        './Tests/BackendIntegrationTests/Constants.swift'
+        './Tests/BackendIntegrationTests/Constants.swift',
+        './Tests/v3LoadShedderIntegration/v3LoadShedderIntegrationTests/V3LoadShedderIntegrationTests.swift'


Would it be worth separating it to a Constants file in the new project as well? Mostly to avoid replacing unintended things

maybe? It feels a bit YAGNI - the whole file has a single constant, and these are integration tests based on a version of the SDK that's in maintenance mode, so it's not like this test suite is really going to grow over time

also, the variable is called REVENUECAT_LOAD_SHEDDER_API_KEY, I don't feel like we're going to be running into a lot of conflicts with it.
I'd vote to just keep it like this, as dead simple as possible, until the day comes where we actually start adding more to this and it becomes harder to manage.

tonidero · 2023-03-30T15:35:55Z

+      output_types: 'junit',
+      output_style: 'raw',
+      result_bundle: true,
+      configuration: 'Debug',


I think running them on Debug is enough for now, since we are mostly just testing the interaction with the load shedder, which should be the same 👍

+1. I'm not sure you even could run them on release since we're using @testable import, but I haven't really checked. Won't make much of a difference in this case anyway.

tonidero · 2023-03-30T15:38:01Z

  end

+  desc "Run LoadShedder tests"
+  lane :v3_loadshedder_integration_tests do |options|


Should we also have a job for v4 to run them on a schedule? We can reuse the existing jobs though, so it's probably ok?

@NachoSoto

### New Features * New `ErrorCode.signatureVerificationFailed` which will be used for an upcoming feature ### Bugfixes * `Purchases.deinit`: don't reset `Purchases.proxyURL` (#2346) via NachoSoto (@NachoSoto) <details> <summary><b>Other Changes</b></summary> * Introduced `Configuration.EntitlementVerificationMode` and `VerificationResult` (#2277) via NachoSoto (@NachoSoto) * `PurchasesDiagnostics`: added step to verify signature verification (#2267) via NachoSoto (@NachoSoto) * `HTTPClient`: added signature validation and introduced `ErrorCode.signatureVerificationFailed` (#2272) via NachoSoto (@NachoSoto) * `ETagManager`: don't use ETags if response verification failed (#2347) via NachoSoto (@NachoSoto) * `Integration Tests`: removed `@preconcurrency import` (#2464) via NachoSoto (@NachoSoto) * Clean up: moved `ReceiptParserTests-Info.plist` out of root (#2460) via NachoSoto (@NachoSoto) * Update `CHANGELOG` (#2461) via NachoSoto (@NachoSoto) * Update `SwiftSnapshotTesting` (#2453) via NachoSoto (@NachoSoto) * Fixed docs (#2432) via Kaunteya Suryawanshi (@kaunteya) * Remove unnecessary line break (#2435) via Andy Boedo (@aboedo) * `ProductEntitlementMapping`: enabled entitlement mapping fetching (#2425) via NachoSoto (@NachoSoto) * `BackendPostReceiptDataTests`: increased timeout to fix flaky test (#2426) via NachoSoto (@NachoSoto) * Updated requirements to drop Xcode 13.x support (#2419) via NachoSoto (@NachoSoto) * `Integration Tests`: fixed flaky errors when loading offerings (#2420) via NachoSoto (@NachoSoto) * `PurchaseTester`: fixed compilation for `internal` entitlement verification (#2417) via NachoSoto (@NachoSoto) * `ETagManager`/`HTTPClient`: sending new `X-RC-Last-Refresh-Time` header (#2373) via NachoSoto (@NachoSoto) * `ETagManager`: don't send validation time if not present (#2490) via NachoSoto (@NachoSoto) * SwiftUI Sample Project: Refactor Package terms method to a computed property (#2405) via Joseph Kokenge (@JOyo246) * Clean up v3 load shedder integration tests (#2402) via Andy Boedo (@aboedo) * Fix iOS 12 compilation (#2394) via NachoSoto (@NachoSoto) * Added new `VerificationResult.verifiedOnDevice` (#2379) via NachoSoto (@NachoSoto) * `PurchaseTester`: fix memory leaks (#2392) via Keita Watanabe (@kitwtnb) * Integration tests: add scheduled job (#2389) via Andy Boedo (@aboedo) * Add lane for running iOS v3 load shedder integration tests (#2388) via Andy Boedo (@aboedo) * iOS v3 load shedder integration tests (#2387) via Andy Boedo (@aboedo) * `Offline Entitlements`: created `LoadShedderIntegrationTests` (#2362) via NachoSoto (@NachoSoto) * Purchases.configure: log warning if attempting to use a static appUserID (#2385) via Mark Villacampa (@MarkVillacampa) * `SubscriberAttributesManagerIntegrationTests`: fixed flaky failures (#2381) via NachoSoto (@NachoSoto) * `@DefaultDecodable.Now`: fixed flaky test (#2374) via NachoSoto (@NachoSoto) * `PurchaseTesterSwiftUI`: fixed iOS compilation (#2376) via NachoSoto (@NachoSoto) * `SubscriberAttributesManagerIntegrationTests`: fixed potential race condition (#2380) via NachoSoto (@NachoSoto) * `Offline Entitlements`: create `CustomerInfo` from offline entitlements (#2358) via NachoSoto (@NachoSoto) * Added `@DefaultDecodable.Now` (#2372) via NachoSoto (@NachoSoto) * `HTTPClient`: debug log when performing redirects (#2371) via NachoSoto (@NachoSoto) * `HTTPClient`: new flag to force server errors (#2370) via NachoSoto (@NachoSoto) * `OfferingsManager`: fixed Xcode 13.x build (#2369) via NachoSoto (@NachoSoto) * `Offline Entitlements`: store `ProductEntitlementMapping` in cache (#2355) via NachoSoto (@NachoSoto) * `Offline Entitlements`: added support for fetching `ProductEntitlementMappingResponse` in `OfflineEntitlementsAPI` (#2353) via NachoSoto (@NachoSoto) * `Offline Entitlements`: created `ProductEntitlementMapping` (#2365) via NachoSoto (@NachoSoto) * Implemented `NetworkError.isServerDown` (#2367) via NachoSoto (@NachoSoto) * `ETagManager`: added test for 304 responses with no etag (#2360) via NachoSoto (@NachoSoto) * `TestLogHandler`: increased default capacity (#2357) via NachoSoto (@NachoSoto) * `OfferingsManager`: moved log to common method to remove hardcoded string (#2363) via NachoSoto (@NachoSoto) * `Offline Entitlements`: created `ProductEntitlementMappingResponse` (#2351) via NachoSoto (@NachoSoto) * `HTTPClient`: added test for 2xx response for request with etag (#2361) via NachoSoto (@NachoSoto) * `PurchaseTesterSwiftUI` improvements (#2345) via NachoSoto (@NachoSoto) * `ConfigureStrings`: fixed double-space typo (#2344) via NachoSoto (@NachoSoto) * `ETagManagerTests`: fixed tests on iOS 12 (#2349) via NachoSoto (@NachoSoto) * `DeviceCache`: simplified constructor (#2354) via NachoSoto (@NachoSoto) * `Trusted Entitlements`: changed all APIs to `internal` (#2350) via NachoSoto (@NachoSoto) * `VerificationResult.notRequested`: removed caching reference (#2337) via NachoSoto (@NachoSoto) * Finished signature verification `HTTPClient` tests (#2333) via NachoSoto (@NachoSoto) * `Configuration.Builder.with(entitlementVerificationMode:)`: improved documentation (#2334) via NachoSoto (@NachoSoto) * `ETagManager`: don't ignore failed etags with `Signing.VerificationMode.informational` (#2331) via NachoSoto (@NachoSoto) * `IdentityManager`: clear `ETagManager` and `DeviceCache` if verification is enabled but cached `CustomerInfo` is not (#2330) via NachoSoto (@NachoSoto) * Made `Configuration.EntitlementVerificationMode.enforced` unavailable (#2329) via NachoSoto (@NachoSoto) * Refactor: reorganized files in new Security and Misc folders (#2326) via NachoSoto (@NachoSoto) * `CustomerInfo`: use same grace period logic for active subscriptions (#2327) via NachoSoto (@NachoSoto) * `HTTPClient`: don't verify 4xx/5xx responses (#2322) via NachoSoto (@NachoSoto) * `EntitlementInfo`: request date is not optional (#2325) via NachoSoto (@NachoSoto) * `CustomerInfo`: removed `entitlementVerification` (#2320) via NachoSoto (@NachoSoto) * Renamed `VerificationResult.notVerified` to `.notRequested` (#2321) via NachoSoto (@NachoSoto) * `EntitlementInfo`: add a grace period limit to outdated entitlements (#2288) via NachoSoto (@NachoSoto) * Update `CustomerInfo.requestDate` from 304 responses (#2310) via NachoSoto (@NachoSoto) * `Signing`: added request time & eTag to signature verification (#2309) via NachoSoto (@NachoSoto) * `HTTPClient`: changed header search to be case-insensitive (#2308) via NachoSoto (@NachoSoto) * `HTTPClient`: automatically add `nonce` based on `HTTPRequest.Path` (#2286) via NachoSoto (@NachoSoto) * `PurchaseTester`: added ability to reload `CustomerInfo` with a custom `CacheFetchPolicy` (#2312) via NachoSoto (@NachoSoto) * Fix issue where underlying error information for product fetch errors was not printed in log. (#2281) via Chris Vasselli (@chrisvasselli) * `PurchaseTester`: added ability to set `Configuration.EntitlementVerificationMode` (#2290) via NachoSoto (@NachoSoto) * SwiftUI: Paywall View should respond to changes on the UserView model (#2297) via ConfusedVorlon (@ConfusedVorlon) * Deprecate `usesStoreKit2IfAvailable` (#2293) via Andy Boedo (@aboedo) * `Signing`: updated to use production public key (#2274) via NachoSoto (@NachoSoto) </details> --------- Co-authored-by: RCGitBot <dev+RCGitBot@revenuecat.com>

@NachoSoto

### New Features * New `ErrorCode.signatureVerificationFailed` which will be used for an upcoming feature ### Bugfixes * `Purchases.deinit`: don't reset `Purchases.proxyURL` (RevenueCat#2346) via NachoSoto (@NachoSoto) <details> <summary><b>Other Changes</b></summary> * Introduced `Configuration.EntitlementVerificationMode` and `VerificationResult` (RevenueCat#2277) via NachoSoto (@NachoSoto) * `PurchasesDiagnostics`: added step to verify signature verification (RevenueCat#2267) via NachoSoto (@NachoSoto) * `HTTPClient`: added signature validation and introduced `ErrorCode.signatureVerificationFailed` (RevenueCat#2272) via NachoSoto (@NachoSoto) * `ETagManager`: don't use ETags if response verification failed (RevenueCat#2347) via NachoSoto (@NachoSoto) * `Integration Tests`: removed `@preconcurrency import` (RevenueCat#2464) via NachoSoto (@NachoSoto) * Clean up: moved `ReceiptParserTests-Info.plist` out of root (RevenueCat#2460) via NachoSoto (@NachoSoto) * Update `CHANGELOG` (RevenueCat#2461) via NachoSoto (@NachoSoto) * Update `SwiftSnapshotTesting` (RevenueCat#2453) via NachoSoto (@NachoSoto) * Fixed docs (RevenueCat#2432) via Kaunteya Suryawanshi (@kaunteya) * Remove unnecessary line break (RevenueCat#2435) via Andy Boedo (@aboedo) * `ProductEntitlementMapping`: enabled entitlement mapping fetching (RevenueCat#2425) via NachoSoto (@NachoSoto) * `BackendPostReceiptDataTests`: increased timeout to fix flaky test (RevenueCat#2426) via NachoSoto (@NachoSoto) * Updated requirements to drop Xcode 13.x support (RevenueCat#2419) via NachoSoto (@NachoSoto) * `Integration Tests`: fixed flaky errors when loading offerings (RevenueCat#2420) via NachoSoto (@NachoSoto) * `PurchaseTester`: fixed compilation for `internal` entitlement verification (RevenueCat#2417) via NachoSoto (@NachoSoto) * `ETagManager`/`HTTPClient`: sending new `X-RC-Last-Refresh-Time` header (RevenueCat#2373) via NachoSoto (@NachoSoto) * `ETagManager`: don't send validation time if not present (RevenueCat#2490) via NachoSoto (@NachoSoto) * SwiftUI Sample Project: Refactor Package terms method to a computed property (RevenueCat#2405) via Joseph Kokenge (@JOyo246) * Clean up v3 load shedder integration tests (RevenueCat#2402) via Andy Boedo (@aboedo) * Fix iOS 12 compilation (RevenueCat#2394) via NachoSoto (@NachoSoto) * Added new `VerificationResult.verifiedOnDevice` (RevenueCat#2379) via NachoSoto (@NachoSoto) * `PurchaseTester`: fix memory leaks (RevenueCat#2392) via Keita Watanabe (@kitwtnb) * Integration tests: add scheduled job (RevenueCat#2389) via Andy Boedo (@aboedo) * Add lane for running iOS v3 load shedder integration tests (RevenueCat#2388) via Andy Boedo (@aboedo) * iOS v3 load shedder integration tests (RevenueCat#2387) via Andy Boedo (@aboedo) * `Offline Entitlements`: created `LoadShedderIntegrationTests` (RevenueCat#2362) via NachoSoto (@NachoSoto) * Purchases.configure: log warning if attempting to use a static appUserID (RevenueCat#2385) via Mark Villacampa (@MarkVillacampa) * `SubscriberAttributesManagerIntegrationTests`: fixed flaky failures (RevenueCat#2381) via NachoSoto (@NachoSoto) * `@DefaultDecodable.Now`: fixed flaky test (RevenueCat#2374) via NachoSoto (@NachoSoto) * `PurchaseTesterSwiftUI`: fixed iOS compilation (RevenueCat#2376) via NachoSoto (@NachoSoto) * `SubscriberAttributesManagerIntegrationTests`: fixed potential race condition (RevenueCat#2380) via NachoSoto (@NachoSoto) * `Offline Entitlements`: create `CustomerInfo` from offline entitlements (RevenueCat#2358) via NachoSoto (@NachoSoto) * Added `@DefaultDecodable.Now` (RevenueCat#2372) via NachoSoto (@NachoSoto) * `HTTPClient`: debug log when performing redirects (RevenueCat#2371) via NachoSoto (@NachoSoto) * `HTTPClient`: new flag to force server errors (RevenueCat#2370) via NachoSoto (@NachoSoto) * `OfferingsManager`: fixed Xcode 13.x build (RevenueCat#2369) via NachoSoto (@NachoSoto) * `Offline Entitlements`: store `ProductEntitlementMapping` in cache (RevenueCat#2355) via NachoSoto (@NachoSoto) * `Offline Entitlements`: added support for fetching `ProductEntitlementMappingResponse` in `OfflineEntitlementsAPI` (RevenueCat#2353) via NachoSoto (@NachoSoto) * `Offline Entitlements`: created `ProductEntitlementMapping` (RevenueCat#2365) via NachoSoto (@NachoSoto) * Implemented `NetworkError.isServerDown` (RevenueCat#2367) via NachoSoto (@NachoSoto) * `ETagManager`: added test for 304 responses with no etag (RevenueCat#2360) via NachoSoto (@NachoSoto) * `TestLogHandler`: increased default capacity (RevenueCat#2357) via NachoSoto (@NachoSoto) * `OfferingsManager`: moved log to common method to remove hardcoded string (RevenueCat#2363) via NachoSoto (@NachoSoto) * `Offline Entitlements`: created `ProductEntitlementMappingResponse` (RevenueCat#2351) via NachoSoto (@NachoSoto) * `HTTPClient`: added test for 2xx response for request with etag (RevenueCat#2361) via NachoSoto (@NachoSoto) * `PurchaseTesterSwiftUI` improvements (RevenueCat#2345) via NachoSoto (@NachoSoto) * `ConfigureStrings`: fixed double-space typo (RevenueCat#2344) via NachoSoto (@NachoSoto) * `ETagManagerTests`: fixed tests on iOS 12 (RevenueCat#2349) via NachoSoto (@NachoSoto) * `DeviceCache`: simplified constructor (RevenueCat#2354) via NachoSoto (@NachoSoto) * `Trusted Entitlements`: changed all APIs to `internal` (RevenueCat#2350) via NachoSoto (@NachoSoto) * `VerificationResult.notRequested`: removed caching reference (RevenueCat#2337) via NachoSoto (@NachoSoto) * Finished signature verification `HTTPClient` tests (RevenueCat#2333) via NachoSoto (@NachoSoto) * `Configuration.Builder.with(entitlementVerificationMode:)`: improved documentation (RevenueCat#2334) via NachoSoto (@NachoSoto) * `ETagManager`: don't ignore failed etags with `Signing.VerificationMode.informational` (RevenueCat#2331) via NachoSoto (@NachoSoto) * `IdentityManager`: clear `ETagManager` and `DeviceCache` if verification is enabled but cached `CustomerInfo` is not (RevenueCat#2330) via NachoSoto (@NachoSoto) * Made `Configuration.EntitlementVerificationMode.enforced` unavailable (RevenueCat#2329) via NachoSoto (@NachoSoto) * Refactor: reorganized files in new Security and Misc folders (RevenueCat#2326) via NachoSoto (@NachoSoto) * `CustomerInfo`: use same grace period logic for active subscriptions (RevenueCat#2327) via NachoSoto (@NachoSoto) * `HTTPClient`: don't verify 4xx/5xx responses (RevenueCat#2322) via NachoSoto (@NachoSoto) * `EntitlementInfo`: request date is not optional (RevenueCat#2325) via NachoSoto (@NachoSoto) * `CustomerInfo`: removed `entitlementVerification` (RevenueCat#2320) via NachoSoto (@NachoSoto) * Renamed `VerificationResult.notVerified` to `.notRequested` (RevenueCat#2321) via NachoSoto (@NachoSoto) * `EntitlementInfo`: add a grace period limit to outdated entitlements (RevenueCat#2288) via NachoSoto (@NachoSoto) * Update `CustomerInfo.requestDate` from 304 responses (RevenueCat#2310) via NachoSoto (@NachoSoto) * `Signing`: added request time & eTag to signature verification (RevenueCat#2309) via NachoSoto (@NachoSoto) * `HTTPClient`: changed header search to be case-insensitive (RevenueCat#2308) via NachoSoto (@NachoSoto) * `HTTPClient`: automatically add `nonce` based on `HTTPRequest.Path` (RevenueCat#2286) via NachoSoto (@NachoSoto) * `PurchaseTester`: added ability to reload `CustomerInfo` with a custom `CacheFetchPolicy` (RevenueCat#2312) via NachoSoto (@NachoSoto) * Fix issue where underlying error information for product fetch errors was not printed in log. (RevenueCat#2281) via Chris Vasselli (@chrisvasselli) * `PurchaseTester`: added ability to set `Configuration.EntitlementVerificationMode` (RevenueCat#2290) via NachoSoto (@NachoSoto) * SwiftUI: Paywall View should respond to changes on the UserView model (RevenueCat#2297) via ConfusedVorlon (@ConfusedVorlon) * Deprecate `usesStoreKit2IfAvailable` (RevenueCat#2293) via Andy Boedo (@aboedo) * `Signing`: updated to use production public key (RevenueCat#2274) via NachoSoto (@NachoSoto) </details> --------- Co-authored-by: RCGitBot <dev+RCGitBot@revenuecat.com>

aboedo added 3 commits March 30, 2023 10:41

added lane to run v3 load shedder integration tests

d58484d

fixed configuration issues

2e2729d

updated name of the environment variable to pick up

fffb3b9

aboedo added the test label Mar 30, 2023

aboedo self-assigned this Mar 30, 2023

aboedo requested a review from a team March 30, 2023 13:52

aboedo commented Mar 30, 2023

View reviewed changes

tonidero approved these changes Mar 30, 2023

View reviewed changes

aboedo merged commit c1c1322 into main Mar 30, 2023

aboedo deleted the andy/sdk-3046-set-up-ios-v3-load-shedder-integration branch March 30, 2023 15:53

RCGitBot mentioned this pull request May 3, 2023

[AUTOMATIC] Release/4.19.0 #2466

Closed

This was referenced May 12, 2023

Release/4.19 #2483

Closed

Release/4.19.0 #2491

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add lane for running iOS v3 load shedder integration tests#2388

Add lane for running iOS v3 load shedder integration tests#2388
aboedo merged 3 commits into
mainfrom
andy/sdk-3046-set-up-ios-v3-load-shedder-integration

aboedo commented Mar 30, 2023

Uh oh!

aboedo Mar 30, 2023

Uh oh!

tonidero Mar 30, 2023

Uh oh!

aboedo Mar 30, 2023

Uh oh!

codecov Bot commented Mar 30, 2023

Uh oh!

tonidero Mar 30, 2023

Uh oh!

aboedo Mar 30, 2023

Uh oh!

aboedo Mar 30, 2023

Uh oh!

tonidero Mar 30, 2023

Uh oh!

aboedo Mar 30, 2023

Uh oh!

tonidero Mar 30, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

aboedo commented Mar 30, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Mar 30, 2023

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants