Skip to content

Conversation

@Youssef1313
Copy link
Member

@Youssef1313 Youssef1313 commented Jun 12, 2025

Fixes #2616

Instead of repeatedly setting Console.Out/Console.Error and resetting to original in a thread-unsafe way, we now set Conosle.Out/Console.Error once in the UnitTestRunner constructor.

Additionally, the logic of capturing is way simplified now.

Each step of the test lifecycle (asm init, asm cleanup, class init, test, class cleanup, and asm cleanup) will set its current TestContext. The current TestContext is async local, so it's isolated properly when running in parallel. Then when each step creates its test result, it just needs to get the data from the current test context.

The redirected out/error will simply write to the current test context if there is one, or write to the original if there isn't one. This simplifies the logic a lot and makes it much easier to follow.

Note that there is a lot of code that can be deleted with this change, but we cannot do so now as it affects public API. I kept the public API the same. We simply just are not using it in the implementation.

@Youssef1313 Youssef1313 marked this pull request as draft June 12, 2025 10:26
@Youssef1313 Youssef1313 force-pushed the dev/ygerges/capture branch 2 times, most recently from 35fedc7 to 1ac0e14 Compare June 12, 2025 10:41
@Youssef1313 Youssef1313 force-pushed the dev/ygerges/capture branch from be4013a to 3543c3e Compare June 12, 2025 16:27
@Youssef1313 Youssef1313 force-pushed the dev/ygerges/capture branch from fb55cd2 to 5a12d2b Compare June 13, 2025 19:12
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 83.65385% with 34 lines in your changes missing coverage. Please review.

Project coverage is 73.79%. Comparing base (2d00b0c) to head (5a12d2b).
Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...PlatformServices/Execution/ConsoleErrorCapturer.cs 9.52% 19 Missing ⚠️
...formServices/Services/TestContextImplementation.cs 88.23% 6 Missing ⚠️
...dapter.PlatformServices/Execution/TestClassInfo.cs 87.50% 5 Missing ⚠️
....PlatformServices/Execution/ClassCleanupManager.cs 33.33% 2 Missing ⚠️
...r.PlatformServices/Execution/ConsoleOutCapturer.cs 95.23% 1 Missing ⚠️
...pter.PlatformServices/Execution/TraceTextWriter.cs 92.30% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5750      +/-   ##
==========================================
- Coverage   73.89%   73.79%   -0.10%     
==========================================
  Files         602      605       +3     
  Lines       37449    36682     -767     
==========================================
- Hits        27672    27070     -602     
+ Misses       9777     9612     -165     
Flag Coverage Δ
Debug 73.79% <83.65%> (-0.10%) ⬇️
integration 73.79% <83.65%> (-0.31%) ⬇️
production 73.79% <83.65%> (-0.10%) ⬇️
unit 73.79% <83.65%> (+4.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ter.PlatformServices/Execution/TestAssemblyInfo.cs 98.98% <100.00%> (+7.14%) ⬆️
...apter.PlatformServices/Execution/TestMethodInfo.cs 90.24% <100.00%> (-0.09%) ⬇️
...ter.PlatformServices/Execution/TestMethodRunner.cs 87.32% <100.00%> (+1.57%) ⬆️
...apter.PlatformServices/Execution/UnitTestRunner.cs 96.18% <100.00%> (+3.68%) ⬆️
...dapter.PlatformServices/PlatformServiceProvider.cs 73.52% <100.00%> (-7.03%) ⬇️
...r.PlatformServices/Execution/ConsoleOutCapturer.cs 95.23% <95.23%> (ø)
...pter.PlatformServices/Execution/TraceTextWriter.cs 92.30% <92.30%> (ø)
....PlatformServices/Execution/ClassCleanupManager.cs 79.31% <33.33%> (-0.69%) ⬇️
...dapter.PlatformServices/Execution/TestClassInfo.cs 80.61% <87.50%> (+1.66%) ⬆️
...formServices/Services/TestContextImplementation.cs 67.51% <88.23%> (-4.99%) ⬇️
... and 1 more

... and 38 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Youssef1313 Youssef1313 marked this pull request as ready for review June 13, 2025 20:08
@nohwnd
Copy link
Member

nohwnd commented Jun 16, 2025

Is there a possibility to prove that this works via a test? All the changed tests seem to just change because of the approach change, but there is no test specific to proving that we can safely write from many threads.

@Evangelink
Copy link
Member

Is there a possibility to prove that this works via a test? All the changed tests seem to just change because of the approach change, but there is no test specific to proving that we can safely write from many threads.

I think we had some example of this but I don't recall where. I think the story was around the out being set by user in parallel of our code running which was causing the _originalXXX field capturing the wrong stream so we were then not restoring the original out.

Copy link
Member

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, adding a test is tricky but some tests were being disabled because of their flakiness related to this issue so we should be able to see if that's enough through multiple runs

@Youssef1313 Youssef1313 merged commit b7abf45 into main Jun 16, 2025
9 checks passed
@Youssef1313 Youssef1313 deleted the dev/ygerges/capture branch June 16, 2025 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TraceListenerManager is not thread safe

5 participants