Skip to content

Integration job error handling#424

Merged
timkimadobe merged 19 commits intoadobe:devfrom
timkimadobe:integration-job-error-handling
Nov 2, 2023
Merged

Integration job error handling#424
timkimadobe merged 19 commits intoadobe:devfrom
timkimadobe:integration-job-error-handling

Conversation

@timkimadobe
Copy link
Copy Markdown
Contributor

@timkimadobe timkimadobe commented Oct 19, 2023

Description

This PR adds more robust categorized error handling for the integration test job by making the following changes:

  1. Add id values for each step so they can be referenced uniquely
  2. Refactors the On failure step to categorize failures into 3 buckets:
    1. exit 1 = integration test failure
    2. exit 2 = workflow setup/process failure
    3. exit 3 = catchall for any unhandled failures
  3. Creates a new step that provides the exit code in JSON format as the step name for the one directly after the on-failure processing
    • This is for the benefit of GitHub API callers, since the actual exit code is not passed through when getting job information from the GitHub API
    • API users should be able to either use:
      1. The step index within the steps array of step objects to get the correct step name - OR
      2. The number field should also be deterministic based on the yaml definition
      • However, this means altering the step position of the JSON name step is a breaking change
    • When failure() is not true - that is, the job passed, the JSON will use the default value of 0

Example snippet of step output from API response:

{
  "name": "{\"exitCode\":2}",
  "status": "completed",
  "conclusion": "success",
  "number": 8,
  "started_at": "2023-10-18T17:22:29.000-07:00",
  "completed_at": "2023-10-18T17:22:29.000-07:00"
},
  1. Updates the run-name property to use the custom ID set in the workflow inputs
    • API users will be able to access this name using the workflow_name key
    • Note: run-name can only reference the github and inputs contexts.
{
  "id": 17842055225,
  "run_id": 6568125852,
  "workflow_name": "My custom ID",
  "head_branch": "integration-job-error-handling",
...
}

Example runs (using my fork)

Successful cron run: https://github.com/timkimadobe/aepsdk-edge-ios/actions/runs/6579230035
Successful run: https://github.com/timkimadobe/aepsdk-edge-ios/actions/runs/6568252546/job/17842380191
Failure with custom ID: https://github.com/timkimadobe/aepsdk-edge-ios/actions/runs/6568126782/job/17842058447
Failure with default name: https://github.com/timkimadobe/aepsdk-edge-ios/actions/runs/6568125852/job/17842055225

I've also set my fork's default branch to the one opening this PR so that the cron + this update hopefully triggers together, since these workflow changes will not be effective as the default behavior in the main repo until it is merged into main

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@codecov
Copy link
Copy Markdown

codecov bot commented Oct 19, 2023

Codecov Report

Merging #424 (df33ea1) into dev (8f46df6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev     #424   +/-   ##
=======================================
  Coverage   96.76%   96.76%           
=======================================
  Files          27       27           
  Lines        1730     1730           
=======================================
  Hits         1674     1674           
  Misses         56       56           

@timkimadobe timkimadobe linked an issue Oct 19, 2023 that may be closed by this pull request
@timkimadobe timkimadobe requested a review from addb October 20, 2023 21:38
@timkimadobe timkimadobe requested a review from addb October 31, 2023 01:22
Copy link
Copy Markdown
Contributor

@addb addb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Let's merge the downmerge PR before merging this.

@timkimadobe timkimadobe merged commit e6d961a into adobe:dev Nov 2, 2023
@timkimadobe timkimadobe deleted the integration-job-error-handling branch November 2, 2023 23:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update integration test workflow for resilience

2 participants