Skip to content

A script for updating Integration Tests thresholds from CI#3443

Merged
Lukasa merged 2 commits intoapple:mainfrom
rnro:update_integration_test_thresholds
Nov 10, 2025
Merged

A script for updating Integration Tests thresholds from CI#3443
Lukasa merged 2 commits intoapple:mainfrom
rnro:update_integration_test_thresholds

Conversation

@rnro
Copy link
Copy Markdown
Contributor

@rnro rnro commented Nov 10, 2025 •
edited
Loading

Motivation:

We have a way to update Benchmarks thresholds from CI logs but not for the older Integration Tests.

Modifications:

A new script which pulls integration test logs and uses our existing parsing script to pull out updated values.

Result:

Easier threshold updates.

❯ ./dev/update-integration-test-thresholds.sh
Usage: ./dev/update-integration-test-thresholds.sh <url>
   or: URL=<url> ./dev/update-integration-test-thresholds.sh

Example:
  ./dev/update-integration-test-thresholds.sh https://github.com/apple/swift-nio/pull/3442
  ./dev/update-integration-test-thresholds.sh https://github.com/apple/swift-nio/actions/runs/19234929677
  URL=https://github.com/apple/swift-nio/pull/3442 ./dev/update-integration-test-thresholds.sh

❯ ./dev/update-integration-test-thresholds.sh 'https://github.com/apple/swift-nio/actions/runs/19234929677/job/54982236271?pr=3442'
** Processing run in apple/swift-nio
** Fetching integration test checks from workflow run 19234929677
** Pulling logs for Swift nightly-main (job 54982236243)
** Updated IntegrationTests/tests_04_performance/Thresholds/nightly-main.json
** Pulling logs for Swift 6.2 (job 54982236259)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.2.json
** Pulling logs for Swift nightly-next (job 54982236267)
** Updated IntegrationTests/tests_04_performance/Thresholds/nightly-next.json
** Pulling logs for Swift 6.0 (job 54982236271)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.0.json
** Pulling logs for Swift 6.1 (job 54982236345)
** Updated IntegrationTests/tests_04_performance/Thresholds/6.1.json
** Done! Updated 5 threshold file(s)

@rnro
A script for updating Integration Tests thresholds from CI
be8132c 
Motivation:

We have a way to update Benchmarks thresholds from CI logs but not for
the older Integration Tests.

Modifications:

A new script which pulls integration test logs and uses our existing
parsing script to pull out updated values.

Result:

Easier threshold updates
@rnro rnro added the semver/none No version bump required. label Nov 10, 2025
@Lukasa Lukasa enabled auto-merge (squash) November 10, 2025 22:13
@Lukasa Lukasa merged commit 2dbd1bf into apple:main Nov 10, 2025
54 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Lukasa Lukasa Lukasa approved these changes

Assignees

No one assigned

Labels

semver/none No version bump required.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@rnro @Lukasa