Skip to content

Detect circular dependencies in factory-based service registrations at runtime#124331

Merged
svick merged 14 commits intomainfrom
copilot/fix-deadlock-in-application-insights
Mar 4, 2026
Merged

Detect circular dependencies in factory-based service registrations at runtime#124331
svick merged 14 commits intomainfrom
copilot/fix-deadlock-in-application-insights

Conversation

Copy link
Contributor

Copilot AI commented Feb 12, 2026

Description

Fixes issue #88390 - Deadlock when configuring ApplicationInsightsServiceOptions due to circular dependencies through factory functions.

Problem

When services are registered using factory functions that call IServiceProvider.GetService(), circular dependencies can cause deadlocks at runtime when the same lock is acquired recursively in CallSiteRuntimeResolver.VisitRootCache().

Solution

Implemented re-entrance detection in VisitRootCache using a [ThreadStatic] HashSet to track call sites currently being resolved on each thread. When a circular dependency is detected (attempting to resolve a call site already in the resolution chain), an InvalidOperationException is thrown with a clear error message instead of deadlocking.

Changes Made

  1. CallSiteRuntimeResolver.cs: Added re-entrance detection

    • Added [ThreadStatic] HashSet<ServiceCallSite> to track call sites being resolved on current thread
    • Uses shared ReferenceEqualityComparer.Instance from CoreLib (leveraging IEqualityComparer contravariance for type safety)
    • Modified VisitRootCache() to:
      • Add call site to tracking set before resolution
      • Check if call site is already being resolved (circular dependency)
      • Remove call site from tracking set in finally block
      • Throw clear exception if circular dependency detected
  2. Microsoft.Extensions.DependencyInjection.csproj: Reference shared ReferenceEqualityComparer

    • Added compile reference to $(CoreLibSharedDir)System\Collections\Generic\ReferenceEqualityComparer.cs
    • Uses well-tested shared implementation from CoreLib
    • Available for non-.NETCoreApp target frameworks via conditional compile
  3. Test Coverage: Added comprehensive tests for factory-based circular dependencies

    • FactoryCircularDependency_DetectedAtResolutionTime - verifies detection at service resolution
    • FactoryCircularDependency_NotDetectedByValidateOnBuild - documents that ValidateOnBuild cannot detect factory cycles
    • Tests verify proper error messages with service type names
    • Uses discard operator (_) for unused values to avoid compiler warnings
  4. Test Helper Classes: Added fake classes for circular dependency testing

    • FactoryCircularDependencyA - Service with constructor dependency on B
    • FactoryCircularDependencyB - Service registered via factory that creates circular dependency

Design Decisions

  • Runtime detection: Detects circular dependencies at resolution time, not during ValidateOnBuild
  • Thread-local tracking: Uses [ThreadStatic] to avoid cross-thread interference
  • Reference equality: Uses shared ReferenceEqualityComparer from CoreLib via contravariance for type-safe tracking
  • No behavioral changes: Doesn't change singleton lifetime semantics or cause side effects
  • Clear error messages: Uses existing SR.CircularDependencyException resource string
  • Code reuse: Leverages shared CoreLib implementation instead of custom code

Benefits

  • Prevents Deadlocks: Circular dependencies throw exceptions instead of causing deadlocks
  • No Side Effects: Doesn't eagerly instantiate singletons
  • Maintains Semantics: Preserves existing singleton lifetime and construction order
  • Works Everywhere: Detects circular dependencies regardless of ValidateOnBuild setting
  • Clear Errors: Developers get actionable error messages with service type names
  • Thread-Safe: Thread-local tracking ensures no interference between concurrent resolutions
  • Type Safety: Uses HashSet<ServiceCallSite> with contravariant IEqualityComparer<object>
  • Code Reuse: Uses shared, well-tested ReferenceEqualityComparer from CoreLib

Testing

  • ✅ Two focused test cases covering factory-based circular dependencies
  • ✅ Tests verify detection at resolution time with proper error messages
  • ✅ Tests document ValidateOnBuild limitations for factory-based cycles
  • ✅ Code compiles successfully with no warnings across all target frameworks
  • ✅ Tests use discard operator to avoid unused variable warnings
  • ✅ Leverages IEqualityComparer contravariance for type safety
Original prompt

This section details on the original issue you should resolve

<issue_title>Deadlock when configuring ApplicationInsightsServiceOptions</issue_title>
<issue_description>### Description

When adding ApplicationInsights to an ASP.NET core web app's IServiceCollection via AddApplicationInsightsTelemetry() and adding a call to ConfigureOptions<ConfigureApplicationInsightsServiceOptions>() to configure ApplicationInsightsServiceOptions, we are experiencing a deadlock.

Sample code

Program.cs:

services.ConfigureOptions<ConfigureApplicationInsightsServiceOptions>();
services.AddApplicationInsightsTelemetry();

ConfigureApplicationInsightsServiceOptions.cs:

public class ConfigureApplicationInsightsServiceOptions : IConfigureOptions<ApplicationInsightsServiceOptions>
    {
        public ConfigureApplicationInsightsServiceOptions(
            SecretClient secretClient)
        {
            SecretClient = secretClient;
        }

        public SecretClient SecretClient { get; }

        public void Configure(ApplicationInsightsServiceOptions options)
        {
            KeyVaultSecret secret = SecretClient.GetSecret("appinsights");
            options.ConnectionString = secret.Value;
        }
    }

Reproduction Steps

  1. Call ConfigureOptions<ConfigureApplicationInsightsServiceOptions>() on the IServiceCollection as shown in the description
  2. Call AddApplicationInsightsTelemetry() on the IServiceCollection as shown in the description

Or see here: https://github.com/jonathan-vogel-siemens/IssueRepro88390

Expected behavior

ConfigureApplicationInsightsServiceOptions.Configure should be invoked when the WebApplicationBuilder is building

Actual behavior

Deadlock. See running tasks here:

image

Somehow it is blocking at

Regression?

Seems like it is a regression. A similar method (yielding the same deadlock) is documented here, from 2021: https://www.uveta.io/categories/blog/unclutter-startup-cs/#Application-Insights

Known Workarounds

No response

Configuration

  • .NET 6.0
  • Windows
  • x64
  • Plain ASP.NET Razor Web App

Other information

No response</issue_description>

<agent_instructions>go and validate the provided repro in the link, don't do anything else, don't implement anything now</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@ Tagging subscribers to this area: @dotnet/area-extensions-options See info in [area-owners.md](https://github.com/dotnet/runtime/blob/main/docs/area-owners.md) if you want to be subscribed.
Issue Details

Description

When adding ApplicationInsights to an ASP.NET core web app's IServiceCollection via AddApplicationInsightsTelemetry() and adding a call to ConfigureOptions<ConfigureApplicationInsightsServiceOptions>(), we are experiencing a deadlock.

Sample code

Program.cs:

services.ConfigureOptions<ConfigureApplicationInsightsServiceOptions>();
services.AddApplicationInsightsTelemetry();

ConfigureApplicationInsightsServiceOptions.cs:

public class ConfigureApplicationInsightsServiceOptions : IConfigureOptions<ApplicationInsightsServiceOptions>
    {
        public ConfigureApplicationInsightsServiceOptions(
            SecretClient secretClient)
        {
            SecretClient = secretClient;
        }

        public SecretClient SecretClient { get; }

        public void Configure(ApplicationInsightsServiceOptions options)
        {
            KeyVaultSecret secret = SecretClient.GetSecret("appinsights");
            options.ConnectionString = secret.Value;
        }
    }

Reproduction Steps

  1. Call ConfigureOptions<ConfigureApplicationInsightsServiceOptions>() on the IServiceCollection as shown in the description
  2. Call AddApplicationInsightsTelemetry() on the IServiceCollection as shown in the description

Expected behavior

ConfigureApplicationInsightsServiceOptions.Configure should be invoked when the WebApplicationBuilder is building

Actual behavior

Deadlock. See running tasks here:

image

Somehow it is blocking at

Regression?

No response

Known Workarounds

No response

Configuration

  • .NET 6.0
  • Windows
  • x64
  • Plain ASP.N...

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@github-actions github-actions bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Feb 12, 2026
Copilot AI changed the title [WIP] Fix deadlock when configuring ApplicationInsightsServiceOptions Validation Complete: No Changes Implemented Feb 12, 2026
Copilot AI requested a review from rosebyte February 12, 2026 09:59
@tarekgh tarekgh added area-Extensions-DependencyInjection and removed needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Feb 12, 2026
@tarekgh
Copy link
Member

tarekgh commented Feb 12, 2026

@copilot detecting circular dependencies at build time (in CallSiteFactory) is sensible and would turn a silent deadlock into a clear exception. However, this is a detection fix, not a resolution fix, it would throw instead of deadlock, but the user's service graph would still be broken. Is that right? apps can work around this by breaking the cycle by injecting IServiceProvider and resolving lazily:

    public ConfigureAppInsightsOptions(IServiceProvider sp)
    {
        _sp = sp;
    }
    
    public void Configure(ApplicationInsightsServiceOptions options)
    {
        var client = _sp.GetRequiredService<SecretClient>();
        options.ConnectionString = client.GetSecret("appinsights").Value;
    }

If decided to do something here, would it make sense to do it with ValidateOnBuild the service oprovider option? Adding a new check in ServiceProvider.ValidateService() or a companion to CallSiteValidator. I know ValidateOnBuild is opt-in behavior but it is enabled by default in aspnetcore in development mode. and apps can decide to do similar things too. This option can catches the problem before it happens. Unlike the proposed fix which try to handle deadlocks at resolution time.

Copy link
Contributor Author

Copilot AI commented Feb 12, 2026

@copilot detecting circular dependencies at build time (in CallSiteFactory) is sensible and would turn a silent deadlock into a clear exception. However, this is a detection fix, not a resolution fix,...

You're correct - I haven't implemented anything yet, only validated the repro per the original instructions.

Your suggestion to add circular dependency detection in ValidateOnBuild makes sense. That would:

  • Catch the issue early in development (enabled by default in ASP.NET Core dev mode)
  • Provide a clear exception before the deadlock occurs
  • Be opt-in for production scenarios

The workaround pattern you mentioned (injecting IServiceProvider for lazy resolution) does break the cycle, though it requires users to understand the issue.

Should I proceed with implementing circular dependency detection in the validation path (ServiceProvider.ValidateService() or CallSiteValidator), rather than at resolution time in CallSiteRuntimeResolver? That would align better with the existing validation infrastructure.

Copilot AI requested a review from tarekgh February 12, 2026 15:13
@tarekgh
Copy link
Member

tarekgh commented Feb 12, 2026

@copilot

Should I proceed with implementing circular dependency detection in the validation path (ServiceProvider.ValidateService() or CallSiteValidator), rather than at resolution time in CallSiteRuntimeResolver? That would align better with the existing validation infrastructure.

Yes, go ahead and do it.

Copy link
Contributor Author

Copilot AI commented Feb 12, 2026

@copilot

Should I proceed with implementing circular dependency detection in the validation path (ServiceProvider.ValidateService() or CallSiteValidator), rather than at resolution time in CallS...

Started implementation. Added test case to reproduce factory-based circular dependency (issue #88390). The test demonstrates that factory functions can hide circular dependencies that aren't detected during call site construction, leading to deadlocks at resolution time.

Copilot AI and others added 3 commits February 12, 2026 15:24
- Added FactoryCircularDependencyA and FactoryCircularDependencyB fake classes
- Added test case FactoryCircularDependency_DetectedWithValidateOnBuild
- Test demonstrates circular dependency through factory functions (issue #88390)
- Currently ValidateOnBuild doesn't detect factory-based circular dependencies

Co-authored-by: tarekgh <10833894+tarekgh@users.noreply.github.com>
Modify ValidateService() to actually resolve singleton services during ValidateOnBuild.
This detects circular dependencies that occur through factory functions (issue #88390).

When a factory function calls IServiceProvider.GetService(), circular dependencies
weren't previously detected during call site construction. Now, by resolving singletons
during validation, these circular dependencies are exposed before they cause deadlocks
at runtime.

Co-authored-by: tarekgh <10833894+tarekgh@users.noreply.github.com>
- Remove unnecessary catch block in ValidateService()
- Add documentation to FactoryCircularDependencyB test helper
- Improve test assertions to specifically check for circular dependency errors
- Test now verifies proper error message in both validateOnBuild modes

Co-authored-by: tarekgh <10833894+tarekgh@users.noreply.github.com>
Copilot AI changed the title Validation Complete: No Changes Implemented Detect circular dependencies in factory-based service registrations during ValidateOnBuild Feb 12, 2026
@tarekgh tarekgh added this to the 11.0.0 milestone Feb 12, 2026
@tarekgh
Copy link
Member

tarekgh commented Feb 12, 2026

@copilot looking more, seems the current approach has fundamental issues as resolving singletons during ValidateOnBuild is wrong because:

The PR adds CallSiteRuntimeResolver.Instance.Resolve(callSite, Root) inside ValidateService() for all singleton services. This eagerly instantiates every singleton at build time, which introduces several problems:

  1. Side effects during validation: Singletons may open database connections, start background threads, make HTTP calls, etc. ValidateOnBuild should only verify the service can be constructed, it should never cause side effects.
  2. Changed singleton lifetime semantics: Singletons get constructed during ValidateOnBuild and cached in callSite.Value. This shifts when singletons come alive and changes observable construction order, a breaking behavioral change.
  3. Ordering issues: Resolving singleton A triggers resolution of its dependencies, which may not have been validated yet, outside the try/catch validation loop.

We'll go back to have the fix in the runtime resolution path instead

Maybe we can implement the re-entrance detection in VisitRootCache using a [ThreadStatic] set:

   [ThreadStatic]
   private static HashSet<ServiceCallSite>? t_resolving;

   protected override object? VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context)
   {
       if (callSite.Value is object value)
           return value;

       var lockType = RuntimeResolverLock.Root;
       ServiceProviderEngineScope serviceProviderEngine = context.Scope.RootProvider.Root;

       lock (callSite)
       {
           if (callSite.Value is object callSiteValue)
               return callSiteValue;

           t_resolving ??= new HashSet<ServiceCallSite>(ReferenceEqualityComparer.Instance);
           if (!t_resolving.Add(callSite))
           {
               throw new InvalidOperationException(
                   SR.Format(SR.CircularDependencyException, callSite.ServiceType));
           }

           try
           {
               object? resolved = VisitCallSiteMain(callSite, new RuntimeResolverContext
               {
                   Scope = serviceProviderEngine,
                   AcquiredLocks = context.AcquiredLocks | lockType
               });
               serviceProviderEngine.CaptureDisposable(resolved);
               callSite.Value = resolved;
               return resolved;
           }
           finally
           {
               t_resolving.Remove(callSite);
           }
       }
   }

Could you check if this approach will work and will not have any side effects?

…tracking

Per @tarekgh's feedback, reverted ValidateOnBuild approach which had fundamental issues:
- Eagerly instantiating singletons caused side effects
- Changed singleton lifetime semantics
- Caused ordering issues

New approach implements re-entrance detection at resolution time:
- Added [ThreadStatic] HashSet to track call sites being resolved on current thread
- Detects circular dependencies in VisitRootCache before deadlock occurs
- Uses custom ReferenceEqualityComparer for ServiceCallSite instances
- Throws InvalidOperationException with clear error message when cycle detected

This addresses issue #88390 without changing validation behavior or singleton semantics.

Co-authored-by: tarekgh <10833894+tarekgh@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 13, 2026

@copilot looking more, seems the current approach has fundamental issues as resolving singletons during ValidateOnBuild is wrong because:

The PR adds CallSiteRuntimeResolver.Instance.Resolve(callSi...

You're absolutely right about the fundamental issues with the ValidateOnBuild approach. I've reverted that and implemented the re-entrance detection in VisitRootCache as you suggested.

Changes in commit aa55803:

  • Added [ThreadStatic] HashSet<ServiceCallSite> to track call sites being resolved
  • Created custom ReferenceEqualityComparer using RuntimeHelpers.GetHashCode() for reference equality
  • Modified VisitRootCache() to detect re-entrance and throw InvalidOperationException before deadlock occurs
  • Uses try/finally to ensure cleanup of tracking set

This approach:

  • ✅ No side effects - doesn't eagerly instantiate services
  • ✅ No semantic changes - preserves singleton lifetime behavior
  • ✅ No ordering issues - detection happens at resolution time
  • ✅ Works across all target frameworks
  • ✅ Detects circular dependencies before deadlock

Copy link
Member

@rosebyte rosebyte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rosebyte
Copy link
Member

rosebyte commented Mar 4, 2026

@copilot , @svick has a good point, go and do what he says in his open comment.

Per @svick's feedback, removed custom ReferenceEqualityComparer and now use the shared
implementation from CoreLib:

- Added reference to $(CoreLibSharedDir)System\Collections\Generic\ReferenceEqualityComparer.cs
- Changed HashSet<ServiceCallSite> to HashSet<object> to work with shared comparer
- Removed custom ReferenceEqualityComparer nested class
- Uses System.Collections.Generic.ReferenceEqualityComparer.Instance

This reduces code duplication and uses the well-tested shared implementation.

Co-authored-by: rosebyte <14963300+rosebyte@users.noreply.github.com>
auto-merge was automatically disabled March 4, 2026 13:47

Head branch was pushed to by a user without write access

Copy link
Contributor Author

Copilot AI commented Mar 4, 2026

@copilot , @svick has a good point, go and do what he says in his open comment.

Done in commit 5a65eb0. Replaced custom ReferenceEqualityComparer with the shared implementation from CoreLib. Added compile reference to $(CoreLibSharedDir)System\Collections\Generic\ReferenceEqualityComparer.cs and updated code to use HashSet<object> with ReferenceEqualityComparer.Instance.

Copilot AI requested a review from rosebyte March 4, 2026 13:49
@svick
Copy link
Member

svick commented Mar 4, 2026

@copilot IEqualityComparer<T> is contravariant, so you should be able to continue using HashSet<ServiceCallSite>.

@rosebyte rosebyte enabled auto-merge (squash) March 4, 2026 13:54
Per @svick's feedback, reverted to HashSet<ServiceCallSite> instead of HashSet<object>.
IEqualityComparer<T> is contravariant, so IEqualityComparer<object> can be used as
IEqualityComparer<ServiceCallSite>, providing better type safety while still using
the shared ReferenceEqualityComparer.Instance from CoreLib.

Co-authored-by: svick <287848+svick@users.noreply.github.com>
auto-merge was automatically disabled March 4, 2026 13:58

Head branch was pushed to by a user without write access

Copy link
Contributor Author

Copilot AI commented Mar 4, 2026

@copilot IEqualityComparer<T> is contravariant, so you should be able to continue using HashSet<ServiceCallSite>.

Excellent point! Fixed in commit f6e3106. Reverted to HashSet<ServiceCallSite> leveraging IEqualityComparer<T> contravariance - IEqualityComparer<object> works as IEqualityComparer<ServiceCallSite>, providing better type safety.

Copilot finished work on behalf of svick March 4, 2026 14:00
Copilot AI requested a review from svick March 4, 2026 14:00
@svick svick enabled auto-merge (squash) March 4, 2026 14:01
@svick svick merged commit 7603a59 into main Mar 4, 2026
90 checks passed
@svick svick deleted the copilot/fix-deadlock-in-application-insights branch March 4, 2026 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deadlock when configuring ApplicationInsightsServiceOptions

6 participants