Retry loop in ContainerReconciler exhausts system RAM #6548
Description
Is there an existing issue for this?
- I have searched the existing issues
Describe the bug
When the container runtime is not healthy, Aspire retries forever with no delay, eventually exhausting system memory.
We recently upgraded to Aspire 9.0 rc1 and started using its ability to manage a Docker container to run Azurite. This morning I tried to run my .net app, but I forgot to pass the run flag that would also start the Azurite container. During the boot process, Aspire continually retried waiting for the container to be healthy, spewing logs to stdout so fast that eventually my Terminal died once it exhausted system memory. The process was using 29GB when I terminated it, nearly all of that consisting of retry log messages (see screenshot)
Expected Behavior
When retrying, it should backoff and/or eventually give up. It should not run a hot loop with no delay.
Steps To Reproduce
No response
Exceptions (if any)
No response
.NET Version info
> dotnet --info
.NET SDK:
Version: 9.0.100-rc.2.24474.11
Commit: 315e1305db
Workload version: 9.0.100-manifests.cea95dba
MSBuild version: 17.12.0-preview-24473-03+fea15fbd1
Runtime Environment:
OS Name: Mac OS X
OS Version: 15.0
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/9.0.100-rc.2.24474.11/
.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.
Host:
Version: 9.0.0-rc.2.24473.5
Architecture: arm64
Commit: 990ebf52fc
.NET SDKs installed:
8.0.304 [/usr/local/share/dotnet/sdk]
9.0.100-rc.2.24474.11 [/usr/local/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0-rc.2.24474.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0-rc.2.24473.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Other architectures found:
None
Environment variables:
Not set
Anything else?
Aspire 9.0 RC1
9.0.0-rc.1.24511.1