Integration testing workflow update by timkimadobe · Pull Request #349 · adobe/aepsdk-edge-ios

timkimadobe · 2023-05-18T22:02:02Z

Description

This PR updates the CircleCI workflow to include a step for integration testing, conditional on the base branch name main or staging

Note that:

Even though the CircleCI env vars CIRCLE_BRANCH (bash context) and << pipeline.git.branch >> (yaml context) are the source branch name, the workflow is run on the destination branch once the PR is merged in

Archive (outdated)
The job build_xcframework_and_app still uses the CircleCI auto populated env var for branch name which means the staging condition will trigger when the main repo's staging branch initiates a PR to any other branch (in our case, main)

This option is simpler since it relies on CircleCI's env vars, but relies on us maintaining strict adherence to our normal branch promotion workflows (dev -> staging, staging -> main)
- The current workflow will also need updates to include regex checks for the "dev" prefix, in case repos want to use "dev-v1.0.0" style branches
- I feel this option could be a bit fragile since it relies on the branch name of the PR initiator to trigger properly

The job test-ios-integration uses GitHub's public API to extract the base branch name which means any PRs opened to merge into the main repo's main or staging will trigger the integration test

No GitHub token is required since the repo the info is being fetched for is public
Personally I like this option more because I feel it aligns better with how we intend the check to work: on any PRs to main or staging, trigger the integration test (and/or xcframeworks build)

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have signed the Adobe Open Source CLA.
My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

Make integration setup steps non conditional

codecov · 2023-05-18T22:14:14Z

Codecov Report

Merging #349 (c50131b) into feature/upstream-integration-tests (a43186b) will not change coverage.
The diff coverage is n/a.

@@                         Coverage Diff                         @@
##           feature/upstream-integration-tests     #349   +/-   ##
===================================================================
  Coverage                               96.73%   96.73%           
===================================================================
  Files                                      27       27           
  Lines                                    1653     1653           
===================================================================
  Hits                                     1599     1599           
  Misses                                     54       54

addb

Looks good!

emdobrin · 2023-05-19T21:27:09Z

Makefile

 	-workspace $(PROJECT_NAME).xcworkspace \
 	-scheme UpstreamIntegrationTests \
-	-destination 'platform=iOS Simulator,name=iPhone 14' \
+	-destination 'platform=iOS Simulator,name=iPhone 8' \


any reason for changing this simulator?

I believe CircleCI runner did not have 14 available?

Please see the job run for the commit before the one to update to 8 in the workflow test PR for reference: https://app.circleci.com/pipelines/github/adobe/aepsdk-edge-ios/982/workflows/1739b0b6-ae18-4735-b944-079684a2f104/jobs/1305

same here, makefile has been updated in dev

Since we've updated our xcode version to 14.1.0 in CircleCI, can you verify if you can use iPhone 14? It appears to be working running the Unit and Functional tests.

Updated to 14!

.circleci/config.yml

timkimadobe

Thanks for the review @emdobrin! Addressed feedback with some outstanding questions

timkimadobe · 2023-05-19T22:09:34Z

Makefile

 	-workspace $(PROJECT_NAME).xcworkspace \
 	-scheme UpstreamIntegrationTests \
-	-destination 'platform=iOS Simulator,name=iPhone 14' \
+	-destination 'platform=iOS Simulator,name=iPhone 8' \


I believe CircleCI runner did not have 14 available?

Please see the job run for the commit before the one to update to 8 in the workflow test PR for reference: https://app.circleci.com/pipelines/github/adobe/aepsdk-edge-ios/982/workflows/1739b0b6-ae18-4735-b944-079684a2f104/jobs/1305

.circleci/config.yml

kevinlind · 2023-06-30T21:20:08Z

.circleci/config.yml

          name: Build XCFramework
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ]; then


Can you include both "main" and "staging" here? There are cases where we may merge directly into main so the code may be different from staging, so it is good to rerun this check.

If the intent is to trigger the build on any PRs trying to merge to main or staging, I think we should use the branch names: staging and dev
This is because CircleCI's env var CIRCLE_BRANCH actually refers to the source branch opening the PR, not the destination branch the PR will merge into (my understanding is that there is no env var out of the box in CircleCI for this today: please see: https://circleci.com/docs/variables/)

So then when dev opens a PR to any other branch (most likely staging as the destination), the build will trigger
and also when staging opens a PR to any other branch (usually main)

But this actually doesn't cover the case where an arbitrary branch opens a PR to merge into main or staging since CIRCLE_BRANCH will be equal to the arbitrary branch name

Please see the thread: https://github.com/adobe/aepsdk-edge-ios/pull/349/files/b0041b14baf8feacf32f7c9d2b9567f067e5069d#r1199418169
For additional context and an alternative way to get destination branch names using the GitHub public API

I also have another thread with additional context I can share privately as well

kevinlind · 2023-06-30T21:20:17Z

.circleci/config.yml

          name: Build Test App
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ]; then


Can you include both "main" and "staging" here? There are cases where we may merge directly into main so the code may be different from staging, so it is good to rerun this check.

Updated! Please see response to above

timkimadobe

Thanks for the review @kevinlind! Added some additional context to the ability to trigger builds on PRs to main or staging branches

Also waiting for #364 to be merged before addressing other feedback

timkimadobe · 2023-07-13T01:15:19Z

.circleci/config.yml

          name: Build XCFramework
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ]; then


If the intent is to trigger the build on any PRs trying to merge to main or staging, I think we should use the branch names: staging and dev
This is because CircleCI's env var CIRCLE_BRANCH actually refers to the source branch opening the PR, not the destination branch the PR will merge into (my understanding is that there is no env var out of the box in CircleCI for this today: please see: https://circleci.com/docs/variables/)

So then when dev opens a PR to any other branch (most likely staging as the destination), the build will trigger
and also when staging opens a PR to any other branch (usually main)

But this actually doesn't cover the case where an arbitrary branch opens a PR to merge into main or staging since CIRCLE_BRANCH will be equal to the arbitrary branch name

Please see the thread: https://github.com/adobe/aepsdk-edge-ios/pull/349/files/b0041b14baf8feacf32f7c9d2b9567f067e5069d#r1199418169
For additional context and an alternative way to get destination branch names using the GitHub public API

I also have another thread with additional context I can share privately as well

timkimadobe · 2023-07-13T01:16:00Z

.circleci/config.yml

          name: Build Test App
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ]; then


Updated! Please see response to above

addb · 2023-07-14T17:03:03Z

.circleci/config.yml

      - test-ios:
          requires:
            - validate-code
+      - test-ios-integration:


Having it in parallel with the tests helps save time, but I am thinking if we should run it after the unit and functional test, so in case those fail we don't run these integration tests. This would help with saving costs and unnecessary e2e run. This may not add much value since we are running on the staging, so ideally all our unit and functional tests would be passing in the dev branch already, but just a thought.

That definitely makes sense to me! I think if we run into a lot of cases where integration testing is eating up a lot of time and unit/functional tests are also failing, we can find a better way to order these tests

…base2

.circleci/config.yml

kevinlind · 2023-07-19T23:57:21Z

Makefile

 	-workspace $(PROJECT_NAME).xcworkspace \
 	-scheme UpstreamIntegrationTests \
-	-destination 'platform=iOS Simulator,name=iPhone 14' \
+	-destination 'platform=iOS Simulator,name=iPhone 8' \


Since we've updated our xcode version to 14.1.0 in CircleCI, can you verify if you can use iPhone 14? It appears to be working running the Unit and Functional tests.

kevinlind · 2023-07-20T00:00:13Z

.circleci/config.yml

          name: Build XCFramework
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ] || [ "${CIRCLE_BRANCH}" == "dev" ]; then


Can you update to run on "main" as well? There can be cases where we merge directly to main so this workflow should get run again.

Currently with the way CircleCI's CIRCLE_BRANCH value is populated - only the source branch that is opening the PR, there's no easy way to get the destination branch (that is, the branch you're merging into)

Please see: https://github.com/adobe/aepsdk-edge-ios/pull/349/files/b0041b14baf8feacf32f7c9d2b9567f067e5069d#r1261868878

Yes, but after the merge happens the target branch is run. The source branch can come from anywhere, usually its staging to main but it could also be a different branch or a branch in a forked repository. So as a sanity check before a release we should run these checks on main.

Oh sorry I didn't know that, thank you for the info! Updated to include the main branch in the check as well

kevinlind · 2023-07-20T00:02:22Z

.circleci/config.yml

          name: Build Test App
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ] || [ "${CIRCLE_BRANCH}" == "dev" ]; then


Can you update to run on "main" as well? There can be cases where we merge directly to main so this workflow should get run again.

Please see above

timkimadobe

Thanks for the reviews @addb, @kevinlind, @emdobrin! Updated based on feedback, and added some additional context

.circleci/config.yml

timkimadobe · 2023-07-20T00:02:09Z

.circleci/config.yml

      - test-ios:
          requires:
            - validate-code
+      - test-ios-integration:


That definitely makes sense to me! I think if we run into a lot of cases where integration testing is eating up a lot of time and unit/functional tests are also failing, we can find a better way to order these tests

timkimadobe · 2023-07-20T00:03:13Z

Makefile

 	-workspace $(PROJECT_NAME).xcworkspace \
 	-scheme UpstreamIntegrationTests \
-	-destination 'platform=iOS Simulator,name=iPhone 14' \
+	-destination 'platform=iOS Simulator,name=iPhone 8' \


Updated to 14!

.circleci/config.yml

timkimadobe · 2023-07-20T00:07:45Z

.circleci/config.yml

          name: Build XCFramework
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ] || [ "${CIRCLE_BRANCH}" == "dev" ]; then


Currently with the way CircleCI's CIRCLE_BRANCH value is populated - only the source branch that is opening the PR, there's no easy way to get the destination branch (that is, the branch you're merging into)

Please see: https://github.com/adobe/aepsdk-edge-ios/pull/349/files/b0041b14baf8feacf32f7c9d2b9567f067e5069d#r1261868878

timkimadobe · 2023-07-20T00:07:53Z

.circleci/config.yml

          name: Build Test App
          command: |
-            if [ "${CIRCLE_BRANCH}" == "main" ]; then
+            if [ "${CIRCLE_BRANCH}" == "staging" ] || [ "${CIRCLE_BRANCH}" == "dev" ]; then


Please see above

timkimadobe · 2023-07-20T00:35:02Z

After discussion:

Since the workflow is run on the destination branch once the PR is merged in, main and staging make the most sense for the conditional jobs
The condition has been moved to the workflow job level using branch filters

Archive (outdated)
Here is an example integration test CI run triggered by this branch name: pull/349 (the way CircleCI populates the branch name env var for this PR):
https://app.circleci.com/pipelines/github/adobe/aepsdk-edge-ios/1114/workflows/4ba55e00-90a5-4728-8933-4f61c9fd1b0e/jobs/1839

I wanted to move the branch name condition to the job level to minimize the setup steps run when the condition is not met (that is, the install_dependencies and prestart_ios_simulator steps)

With the current workflow, the integration test job would also trigger when dev and staging branches open PRs to other branches

I will roll back the test branch condition now that the workflow trigger in CircleCI using this approach has been demonstrated

Makefile

emdobrin · 2023-07-20T01:31:04Z

.circleci/config.yml

+      - when:
+          condition:
+            or:
+              - equal: [ staging, << pipeline.git.branch >> ]
+              - equal: [ dev, << pipeline.git.branch >> ]


should we use the same CIRCLE_BRANCH similar to the other targets?

Conditional check moved to the workflow job level using job branch filters!

~~I was thinking to move the branch name condition to the job level to minimize the setup steps run when the condition is not met (that is, the install_dependencies and prestart_ios_simulator steps)~~

~~However, I can revert it back to the same style check as the bash if check in line with the other CIRCLE_BRANCH checks if that's preferred~~

~~Please see additional context: #349 (comment)~~

…r tests Add removal of existing old results part of the job like other tests

timkimadobe

Thanks for the review @kevinlind and @emdobrin! Updated based on feedback

timkimadobe · 2023-07-20T03:09:05Z

.circleci/config.yml

+      - when:
+          condition:
+            or:
+              - equal: [ staging, << pipeline.git.branch >> ]
+              - equal: [ dev, << pipeline.git.branch >> ]


Conditional check moved to the workflow job level using job branch filters!

~~I was thinking to move the branch name condition to the job level to minimize the setup steps run when the condition is not met (that is, the install_dependencies and prestart_ios_simulator steps)~~

~~However, I can revert it back to the same style check as the bash if check in line with the other CIRCLE_BRANCH checks if that's preferred~~

~~Please see additional context: #349 (comment)~~

Makefile

kevinlind · 2023-07-20T22:01:32Z

.circleci/config.yml

+              - equal: [ staging, << pipeline.git.branch >> ]
+              - equal: [ dev, << pipeline.git.branch >> ]


Should we include "main" here as well so the integration tests run when we merge to "main"?

Conditional check moved to the workflow job level using job branch filters!

~~With the way CircleCI populates the << pipeline.git.branch >> there is no way to detect a merge to main that I could find out of the box~~

…rkflow job branch filter

* Edge Network (Konductor) integration test with GitHub Action workflow (#321) * initial e2e test * update pods for e2e test target * update workflow config settings * implement pods cache to prevent high macos usage time * Fix method typo in functional test helper * test for locationHint * Update integration test case * Remove specific hint value * update action to include build cache step * update buildcache to upload logs for debug * fix flag typo * trying different cache system * try with command line settings * update to include key * fix var bug in makefile command * remove playground * Remove build cache testing * passing env vars to test target working * Update e2e flow action, yaml, and test scheme for konductor env * Fix action command to updated name * Updating action text and defaults * Update make command for e2e test to improve documentation add conditional check warning output for KONDUCTOR_ENV * update test names * update job name * Update env var extraction logic * update test to use edge location hint * update action to have location hint options * fix github action var name * Update makefile env vars and include edge location hint * update makefile integration test docs test colon in test run echo update env var extraction documentation in test file * update test output to use raw string values * test using shell if else for job run id * fix spacing syntax * Update to user proper Edge Network name * Update to EDGE_ENVIRONMENT * Small doc updates * Update documentation to correct links * Update dropdown description * Add quiet flag to xcodebuild command for integration test * Rename integration test make command Reorder position to under tvOS test * Update name to UpstreamIntegrationTests Test empty string option for dropdown * Update action script to use updated make command * Split enums into separate file, add env var extraction inits * run pod install after new target name * Remove FunctionalTestBase inheritance and unused compile sources * Add validation for network request portion * Rename and simplify IntegrationTestNetworkService to only required methods and properties * Update to use raw multiline string instead of resource files * Revert podfile lock version upgrade * Update action job name to align with makefile name * Update target name in makefile * Remove extra newlines Remove marketing version from integration test debug target build settings * Remove implementation specific documentation * Test spacing in makefile * Test remove silent option * test moving back to the bottom * test updating command name * test newline * test name change again * Revert all test name changes - the workflow error was not specifying the correct branch to run off of * Add job failure help text * Add doc comment to on failure step * Revert changes in functional test utils * Split test enums into separate files with shared util method * Clean up comments and update test method * Update EdgeEnvironment enum to have .prod default value Update usage site in test setup * Update to include xcresult visualizer tool * Update with example test failure case * Update env file ID to use helper method Move NetworkTestingDelegate to network file * Omit coverage from test results to save characters * Fix clang flag warning from cocoapods * Remove Xcode report tool step from this PR * Update result bundle path to avoid name conflicts with other tests * Add testing documentation comment hint * Flexible JSON comparison system (#332) * Implement JSON comparison system * Update to use AEPServices AnyCodable * Move assertion helpers to AnyCodableUtils Update assertion methods to accept file and line args for inline errors on test failure Update key path logic to pretty print * Create flexible validation system, with exact match pathing * Exact match wildcard example * Update general wildcard logic to apply to all elements Rename arguments and usage sites * Complete revamp of flexible json comparison system * Convert AnyCodable test assertion helpers to protocol with default implementations Update test classes to adhere to new protocol and update usages Update flexible assertion to use single method with parameter for default mode Clean up code and update documentation for helper methods Fix bug with escaped keys and add unit test case * Apply swift lint * Extract AnyCodable array extension into separate file * Remove unused test setup methods * Remove redundant test cases covered by AnyCodable unit tests * Switch from protocol to XCTestCase extension Update usages Update documentation text * Update keyPathAsString signature and usages * Simplify implementation of AnyCodable array comparison * Remove unused EventSpec * Update filenames Remove Bool return from assertEqual methods Refactor to allow single public API for assertEqual * Update flexible comparison APIs * Add additional test cases for AnyCodable unit tests * Apply swift lint formatting * Update to use positive case * Update method order Update method signature styling * Update actual condition * Update flexible text setup to propagate file and line Upgrade getCapturedRegexGroups to a test failure Upgrade alternate path index errors to TEST ERROR, add file and line * Fix incorrect find and replace * Update regex capture to handle special cases Add test cases that validate special cases * Extract regex logic into shared function * Update shared testing utilities for integration tests (#334) * Move shared test files * Refactor FunctionalTestBase and FunctionalTestNetworkService to allow for dual mode This allows for sharing common test utilities between functional and integration tests * Renaming all usages of FunctionalTestBase to updated name * Rename mock mode bool * Update method docs and cleanup code comments * Update initializer param Create separate data structs for mocked and real network responses Update connectAsync logic structure * Move XCTestCase+AnyCodableAsserts to shared test utils Update method docs for TestNetworkService Update mock bool in TestBase Move AnyCodable helper methods to shared test utils * Update TestConstants name and usages * Consolidate networkResponses into single data struct With mocked behavior controlled by mockNetworkService flag * Update FunctionalTestConst usages * Temporarily moving TestNetworkService for review * Update set get logic for network responses Consolidate awaitRequest logic Simplify NetworkRequest construction Use force unwrap for test constructions * Move NetworkRequest extension to FTNS * Integration tests - TestNetworkService mock and server API split (#337) * Move FTNS to shared test utils * Rename FTNS to TNS * Initial split of mock and server test network service implementations Before removal of network logic from TestBase * Migrate TestBase network APIs to TestNetworkService Move mock or server specific APIs to respective child classes * Update log source name Update reset test expectations logic * Remove resetting network service logic from test base * Rename delayed network response param and var Remove outdated method docs for return value * Update setup logic Update reset test expectations usage * Update network service to static immutable var Update usages to Self * Remove todo as task is wont do * Update mock response API param name * Update integration test to use static network service Also add helper comments to the purpose of the different setup methods * Rename Server to Real TNS Update base class inheritance for both mock and real TNS Update base TNS to be a shared network request helper Update usages within mock and real TNS to use new shared helper as instance var Add passthrough APIs for both mock and real TNS to access helper methods as needed Refactor static networkService in test class to instance to remove need for Self prefix Update associated setUp logic Add mock prefix to networkService in functional test for clarity * Rename mock and real network service classes * Clean up implementation Update doc comment Add implementation note for isEqual * Move shared network service logic to helper Update usage sites Remove unused import * Remove unused imports * Move NetworkRequest flatten body method into NetworkRequest extension Update usages * Update access level * Update doc class names * Remove unneeded commented code * Move testbase debug flag to per case setup Update doc comment * Refactor CompletionHandlerFunctionalTests to use MockNetworkService * Refactor EdgeConsentTests to use MockNetworkService * Add networkService reset * Add NetworkService reset * Refactor EdgePublicAPITests to use MockNetworkService * Refactor AEPEdgePathOverwriteTests Add test flow doc comments * Refactor IdentityStateFunctionalTests Move debug flag set to after setup * Refactor NoConfigFunctionalTests Update doc comment for what method is used to determine equality * Refactor SampleFunctionalTests * Apply lint autocorrect to PR files * Remove unneeded commented code * Integration test cases (#346) * Implementation notes * WIP invalid datastream test * Add test cases Complex XDM, complex data Preset location hint expected error dataset ID expected error invalid location hint * Update setExpectation API to accept only NetworkRequest Update docs Update usages of updated API Update networkService API usages in integration test class * Clean up code comments * Fix for unpassed params * Update class docs Add test note on how JSON comparison system works * Apply swift lint autocorrect * Rename vars and add additional assert on response count Add changes lost in rebase * Updated flexible JSON comparison notes Refactored assertEdgeResonseHandle API and usages Refactored location hint test case to use single variable for location hint value * Update network service class docs * Update first test case to have two event validation * Remove unused API and refactor used API to call helper directly * Update assertEdgeResponseHandle signature and usages Create new helper methods to construct interact URLs with location hint * Add strict count assertion for all responses Update matchedResponses name to be uniform across test suite * Update assertEdgeResponseError method to use getDispatchedEventsWith directly Revert refactor of assertEdgeResponseHandle and usages Add expectedCount argument to both APIs and update assertion logic to check all events Add org ID related exact match assertions and exact match paths (non-wildcard) Add exact match requirement for error type URL * Remove unused API (actually) * Update test cases to use direct assertions * Refactor assert*Match APIs to non-nil expected AnyCodable * Update assertExpectedEvents API to allow for custom timeout period * Update to use conditional check on location hint if initial value is set Simplify 2x event handle test case Add longer timeout for event expectations * Update test case setup Remove outdated API comment * Remove unused helper API * Refactor functional tests from dev branch to use new testing utilities * Use longer extension registration timeout * Add per test case teardown network service reset * Test extending teardown sleep duration * Add higher timeout value for startup event validation * Add sleep to allow more buffer for listener registration * Extend setup timeout to 10s * Restore original teardown sleep time * Remove sleep from test case * Change order of operations for test teardown * Test unregistering instrumented extension in teardown process * Add sleep to startup process to allow event hub setup time * Add mock network service teardown to all functional tests * Trigger CI workflow * Revert changes in TestBase teardown * Integration testing workflow update (#349) * Add new integration test job * Update device for integration to 8 * Update integration test job name * Remove extra space * Remove code coverage upload from integration job * Test job conditional in circleci workflow * Add non conditional step * Update triggering branches * Update initiating branch name condition to staging * test fetching branch name from fork * remove spaces from command * Test extract both base and head branch names * Add semicolons * Update extraction commands * Only fetch base branch name Make integration setup steps non conditional * Update job trigger conditions * Update simulator for integration test to 14 * Update integration test workflow to use conditional at job level * Update integration test job Xcode version * Fix conditional in integration test job * Move checkout step to non-conditional level * Test current branch name also triggers integration test workflow * Test branch name 2 * Remove test branch name * Add main branch to check * Update deployment version for integration target to 11 * Make integration test make command result output consistent with other tests Add removal of existing old results part of the job like other tests * Refactor to remove branch name conditionals from job level and use workflow job branch filter * Test workflow job filter when set on current branch * Update filter criteria * Remove test branch filter * Reorder jobs so conditional ones come after mandatory * Update integration test target in podfile --------- Co-authored-by: Emilia Dobrin <33132425+emdobrin@users.noreply.github.com>

timkimadobe added 15 commits May 18, 2023 14:59

Add new integration test job

2d5a65d

Update device for integration to 8

a23fbe9

Update integration test job name

c544302

Remove extra space

f823f29

Remove code coverage upload from integration job

f12eaa4

Test job conditional in circleci workflow

e92ff72

Add non conditional step

0db4573

Update triggering branches

c924398

Update initiating branch name condition to staging

0f6c9a7

test fetching branch name from fork

24addb1

remove spaces from command

f00eec3

Test extract both base and head branch names

7e2e246

Add semicolons

94eca14

Update extraction commands

bd7f41a

Only fetch base branch name

b0041b1

Make integration setup steps non conditional

timkimadobe linked an issue May 18, 2023 that may be closed by this pull request

Add integration testing workflow to staging and main PR check flow #331

Closed

timkimadobe requested review from addb, cacheung, emdobrin and kevinlind May 18, 2023 23:00

addb approved these changes May 19, 2023

View reviewed changes

emdobrin reviewed May 19, 2023

View reviewed changes

timkimadobe commented May 19, 2023

View reviewed changes

kevinlind reviewed Jun 30, 2023

View reviewed changes

Update job trigger conditions

6aa831a

timkimadobe commented Jul 13, 2023

View reviewed changes

timkimadobe requested review from addb, emdobrin and kevinlind July 13, 2023 01:31

addb reviewed Jul 14, 2023

View reviewed changes

Merge branch 'feature/upstream-integration-tests' into it-workflow-re…

9beb1c8

…base2

kevinlind reviewed Jul 20, 2023

View reviewed changes

timkimadobe added 3 commits July 19, 2023 17:03

Update simulator for integration test to 14

fb3f920

Update integration test workflow to use conditional at job level

173fcb2

Update integration test job Xcode version

836ea77

timkimadobe commented Jul 20, 2023

View reviewed changes

timkimadobe added 4 commits July 19, 2023 17:12

Fix conditional in integration test job

b291ded

Move checkout step to non-conditional level

fc45c0a

Test current branch name also triggers integration test workflow

c470d33

Test branch name 2

57d86e4

Remove test branch name

1bf7430

emdobrin reviewed Jul 20, 2023

View reviewed changes

timkimadobe added 3 commits July 19, 2023 20:07

Add main branch to check

800eac7

Update deployment version for integration target to 11

11ed898

Make integration test make command result output consistent with othe…

129da8d

…r tests Add removal of existing old results part of the job like other tests

timkimadobe commented Jul 20, 2023

View reviewed changes

kevinlind reviewed Jul 20, 2023

View reviewed changes

timkimadobe added 5 commits July 20, 2023 16:52

Refactor to remove branch name conditionals from job level and use wo…

4eeef32

…rkflow job branch filter

Test workflow job filter when set on current branch

1725caa

Update filter criteria

14bc098

Remove test branch filter

269882d

Reorder jobs so conditional ones come after mandatory

c50131b

kevinlind approved these changes Jul 21, 2023

View reviewed changes

timkimadobe merged commit b140354 into adobe:feature/upstream-integration-tests Jul 21, 2023

timkimadobe deleted the it-workflow-rebase2 branch July 25, 2023 21:45

		- equal: [ staging, << pipeline.git.branch >> ]
		- equal: [ dev, << pipeline.git.branch >> ]

Conversation

timkimadobe commented May 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Uh oh!

codecov bot commented May 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

addb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timkimadobe left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timkimadobe left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timkimadobe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

timkimadobe commented May 18, 2023 •

edited

Loading

codecov bot commented May 18, 2023 •

edited

Loading

timkimadobe left a comment •

edited

Loading

timkimadobe commented Jul 20, 2023 •

edited

Loading

timkimadobe Jul 20, 2023 •

edited

Loading

timkimadobe Jul 20, 2023 •

edited

Loading

timkimadobe Jul 20, 2023 •

edited

Loading