Skip to content

Fixes a race condition in Config Admin shutdown#1043

Merged
tcormackMW merged 4 commits intodevelopmentfrom
fixForCAShutdown
Oct 9, 2024
Merged

Fixes a race condition in Config Admin shutdown#1043
tcormackMW merged 4 commits intodevelopmentfrom
fixForCAShutdown

Conversation

@tcormackMW
Copy link
Copy Markdown
Contributor

@tcormackMW tcormackMW commented Oct 2, 2024

In the current implementation, the ConfigurationAdminImpl object is being destroyed from a thread posted to the AsyncWorkService. This occurs because the ConfigurationListenerImpl holds the last reference to configAdmin. When ConfigurationListenerImpl completes its task of notifying listeners, it releases its reference, triggering the destruction of configAdminImpl. During its destruction, configAdminImpl waits for all its posted tasks to complete, including the NotifyConfigurationUpdated tasks in the AsyncWorkService (which this thread itself is), leading to a deadlock as these tasks can never complete.

Solution
To resolve this issue, I modified the Activator::Stop method for ConfigAdminImpl. Instead of setting configAdminImpl to nullptr immediately after unregistering, ensure that configAdminImpl stops posting new asynchronous work and waits for all existing tasks to complete. This ensures that ConfigurationListenerImpl does not hold the last reference to configAdmin, preventing its premature destruction and avoiding the deadlock

Test Case:
Before change: deadlocks roughly every 1/300 runs
After change: didn't deadlock after 120,000 runs

Comment thread compendium/ConfigurationAdmin/test/TestAsyncWorkService.cpp
@codecov
Copy link
Copy Markdown

codecov bot commented Oct 2, 2024

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Project coverage is 88.40%. Comparing base (2e4eca4) to head (9594b30).
Report is 1 commits behind head on development.

Files with missing lines Patch % Lines
.../ConfigurationAdmin/src/ConfigurationAdminImpl.cpp 91.66% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #1043      +/-   ##
===============================================
- Coverage        88.42%   88.40%   -0.03%     
===============================================
  Files              252      252              
  Lines            12155    12165      +10     
===============================================
+ Hits             10748    10754       +6     
- Misses            1407     1411       +4     
Files with missing lines Coverage Δ
compendium/ConfigurationAdmin/src/CMActivator.cpp 89.15% <100.00%> (+0.13%) ⬆️
.../ConfigurationAdmin/src/ConfigurationAdminImpl.hpp 100.00% <100.00%> (ø)
.../ConfigurationAdmin/src/ConfigurationAdminImpl.cpp 90.94% <91.66%> (-0.25%) ⬇️

... and 1 file with indirect coverage changes

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread compendium/ConfigurationAdmin/test/TestAsyncWorkService.cpp
Comment thread compendium/ConfigurationAdmin/test/TestAsyncWorkService.cpp Outdated
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread compendium/ConfigurationAdmin/test/TestAsyncWorkService.cpp
@tcormackMW tcormackMW self-assigned this Oct 8, 2024
Comment thread compendium/ConfigurationAdmin/src/ConfigurationAdminImpl.cpp Outdated
Comment thread compendium/ConfigurationAdmin/src/ConfigurationAdminImpl.hpp Outdated
Comment thread compendium/ConfigurationAdmin/test/TestAsyncWorkService.cpp
@tcormackMW tcormackMW merged commit 2b1206e into development Oct 9, 2024
@tcormackMW tcormackMW deleted the fixForCAShutdown branch October 9, 2024 15:19
Copy link
Copy Markdown
Contributor

@Burgch Burgch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this Toby!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants