Conversation
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 14928Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 14928" |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes flakiness in OpenAIFunctionalTests.DependentResourceWaitsForOpenAIModelResourceWithHealthCheckToBeHealthy by preventing the model resource from inheriting the parent OpenAI resource’s default health check that triggers external HTTP calls during test execution.
Changes:
- Refactors the test to keep a handle to the parent OpenAI resource builder.
- Removes the parent OpenAI resource’s default
"resource_check"health check annotation before building the app to avoid external calls tostatus.openai.com.
JamesNK
reviewed
Mar 4, 2026
Comment on lines
+34
to
+37
| // Remove the default status page health check from the parent OpenAI resource | ||
| // to avoid external HTTP calls to status.openai.com during tests. | ||
| var statusPageHealthCheck = Assert.Single(openai.Resource.Annotations, x => x is HealthCheckAnnotation hca && hca.Key == "resource_check"); | ||
| openai.Resource.Annotations.Remove(statusPageHealthCheck); |
Member
There was a problem hiding this comment.
How about removing everything other than blocking_check? That's the only one this test cares about.
Member
There was a problem hiding this comment.
What about DependentResourceWaitsForOpenAIResourceWithHealthCheckToBeHealthy?
Member
Author
There was a problem hiding this comment.
@copilot respond to the feedback, and handle that.
Member
There was a problem hiding this comment.
How about removing everything other than blocking_check? That's the only one this test cares about.
Is this statement even true?
If the idea of the tests is to run OAI health checks, and the OAI health checks then ping the OAI servers, then these tests are inheritanly flaky depending on network access to OAI and OAI being up
…HTTP calls The OpenAI model resource inherits the parent resource's health check via TryGetAnnotationsIncludingAncestorsOfType. The parent's resource_check calls https://status.openai.com/api/v2/status.json which can timeout in CI, preventing the model from reaching healthy status. Remove the parent's status page health check in the test, matching the pattern already used by the non-quarantined sibling test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
826e6dc to
7a8a73c
Compare
Contributor
16 tasks
Contributor
🎬 CLI E2E Test RecordingsThe following terminal recordings are available for commit
📹 Recordings uploaded automatically from CI run #22652814908 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Flaky Test Fix
Test
Aspire.Hosting.OpenAI.Tests.OpenAIFunctionalTests.DependentResourceWaitsForOpenAIModelResourceWithHealthCheckToBeHealthyRoot Cause
The
OpenAIModelResourceimplementsIResourceWithParent<OpenAIResource>, soResourceHealthCheckServicediscovers health checks from both the model and its parent viaTryGetAnnotationsIncludingAncestorsOfType<HealthCheckAnnotation>(). The parent'sresource_checkhealth check makes an external HTTP call tohttps://status.openai.com/api/v2/status.json, which can timeout or fail in CI, preventing the model resource from ever reaching healthy status.Fix
Remove the parent's
resource_checkhealth check annotation in the test before building the app, matching the pattern already used by the non-quarantined sibling test (DependentResourceWaitsForOpenAIResourceWithHealthCheckToBeHealthy).Verification
Local runs:
Verification Rationale
High confidence — root cause is a clear pattern match (external HTTP calls in health check inherited by child resource). The sibling test already demonstrates the correct fix pattern. CI verification confirms 50/50 pass on Linux, the most affected OS (14% failure rate pre-fix).
Notes
[QuarantinedTest]attribute kept — unquarantining happens separately after 21 days of zero failuresThis fix was generated using the fix-flaky-test skill.