Fix our x86 test execution by jaredpar · Pull Request #60323 · dotnet/roslyn

jaredpar · 2022-03-22T20:18:06Z

The transition to dotnet test seems to have broken our unit tests that
depend on executing on an x86 architecture. This fixes that by using the
--arch argument to the dotnet test execution.

This was made more general to facilitate our future arm64 work in this
area

The transition to `dotnet test` seems to have broken our unit tests that depend on executing on an x86 architecture. This fixes that by using the `--arch` argument to the `dotnet test` execution. This was made more general to facilitate our future arm64 work in this area

jaredpar · 2022-03-22T20:18:25Z

Verified locally this does cause the x86 test runner to be used in -testArch x86

jaredpar · 2022-03-22T20:18:34Z

@dotnet/roslyn-infrastructure PTAL

RikkiGibson · 2022-03-22T20:20:50Z

Is there anything else we should do to verify here? For example, temporarily insert a test which has different behaviors depending on sizeof(nint) and check that we get the expected behavior in each test job?

jaredpar · 2022-03-22T20:27:17Z

Is there anything else we should do to verify here? For example, temporarily insert a test which has different behaviors depending on sizeof(nint) and check that we get the expected behavior in each test job?

I could not think of a simple way to verify this on x86 / x64. The tests are only compiled once hence not clear how we can have tests that validate specific behaviors that we can query on.

The good news is that once we go to arm64 this gets trivial to verify. Our test leg times out even when running minimal tests if we accidentally use x64 emulation

RikkiGibson · 2022-03-22T20:47:57Z

I opened #60324 to test in the way I described. I don't really anticipate having an ongoing test for this, it seems too much like "assert that X is X", but thought a sanity test at the time we introduce it would be useful.

jaredpar · 2022-03-22T22:16:12Z

Great news, I was wrong. Our tests were actually successfully executing on x86.
Bad news, they were not executing on x64 ... ever

src/EditorFeatures/CSharpTest2/EmbeddedLanguages/Json/CSharpJsonTests.cs

sharwell · 2022-03-23T00:15:11Z

src/Tools/Source/RunTests/ProcessTestExecutor.cs

                }
            }

+            builder.Append($@" --arch {assemblyInfo.Architecture}");


src/EditorFeatures/CSharpTest2/EmbeddedLanguages/Json/CSharpJsonTests.cs

Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com>

jcouv · 2022-03-23T17:09:36Z

eng/build.ps1

  Write-Host "Test actions"
-  Write-Host "  -test32                   Run unit tests in the 32-bit runner"
-  Write-Host "  -test64                   Run units tests in the 64-bit runner"
+  Write-Host "  -testArch                 Maps to --arch parameter of dotnet test"


Consider documenting the accepted values

I wanted to leave that out because the set of values isn't constant. It's whatever the underlying dotnet test supports and that changes across versions.

jcouv · 2022-03-23T17:12:48Z

src/Compilers/VisualBasic/Test/Emit/CodeGen/CodeGenTests.vb

        End Sub

-        <ConditionalFact(GetType(NoIOperationValidation), GetType(WindowsOnly))>
+        <ConditionalFact(GetType(NoIOperationValidation), GetType(WindowsOnly), GetType(IsEnglishLocal))>


Why is this needed?

Now that we are actually running on x64 this stress test is failing on Spanish x64. The tolerance on Spanish x64 is stronger than English x64 for reasons that I don't fully understand.

At the moment I'm just trying to get this PR green. This test doesn't change code, it just makes our x64 tests actually run on x64. Fixing the easy bugs I find but overall just trying to get us to green. If this gets us to green my intent is going to file a bug here and merge.

jcouv · 2022-03-23T17:22:01Z

broken our unit tests

How broken were they?

jcouv

LGTM Thanks (iteration 5)

jaredpar · 2022-03-23T17:37:29Z

@jcouv

How broken were they?

It more broke our infra that expected to be running tests on a variety of x86 and x64. It did allow a handful of tests to be written that executed correctly on x86 but failed on x64. Thus far all test bugs, no product bugs. Nevertheless it's tedious to work through the failures to get to a passing state again.

…red-members * upstream/main: (50 commits) Fix top-level speculation (dotnet#60274) Skip CodeDefinitionWindows integration tests Avoid resizing array builder while get unaliased references (dotnet#60220) Fix our x86 test execution (dotnet#60323) [LSP] Don't adjust breakpoints when there are diagnostics in the document (dotnet#60334) Allow VS Mac to access a few options and a helper method (dotnet#60333) Add support for CheckedStatement for smart break line (dotnet#60308) Initialize MEF parts before switching to the UI thread (dotnet#60289) Don't offer convert to switch when an implicit cast is present (dotnet#60224) Fix Add basic TS LSP server (dotnet#59639) Fix issue when multiple different diagnostics share a single fixer Update dependencies from https://github.com/dotnet/arcade build 20220321.2 (dotnet#60315) Check for reference assembly before deciding which language to show (dotnet#60271) Fix XLF files that were manually edited Revert "Fix a typo and an inconsistency" Fix conversion issues in null coalescing operator (dotnet#60202) Fix type parameter scoping for local functions (dotnet#60098) Update SDK version in docs, etc. (dotnet#60299) Edit the breaking change doc for publishing on docs (dotnet#60260) ...

* Create solution on the main thread in integration tests * Close solution on the main thread in integration tests * Use cached data for completion of unimported types Even if it might be stale. the completion list would be computed with cached data and cache refresh would be done in background. * Use cached data for completion of unimported extension methods Even if it might be stale. the completion list would be computed with cached data and cache refresh would be done in background. * Try to refresh import completion cache in the background whenever completion is triggered * Use AsyncBatchingWorkQueue for background cache updates * Fix tests * clean up * Fix tests * Make sure background task to update cache always run * Expose ForceExpandedCompletionIndexCreation option to O# This can be used to tweak test behavior to make it deterministic * Don't use Workspace.CurrentSolution when refreshing cache in background * Use AsynchronousOperationListener for background workqueues * Update src/Tools/ExternalAccess/OmniSharp/Completion/OmniSharpCompletionOptions.cs Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com> * Pass ThreadingContext.DisposalToken to workqueue for cache refresh * Remove namespace imports * Remove enableRoslynAnalyzers from codespace default settings Follow up to #60165 We don't think this is the right default as users probably don't want this unless they have at least 8 cores. * Fix typo in Apply_namespace_matches_folder_preferences * Remove period at the end to be consistent with other messages * Fix inconsistency in other languages too * Refactoring * Ban incompatible service provider extension methods * Assert window title before closing * Remove outdated comment (#60275) * Don't block on clearing the output window on the UI thread when generating a file (#60281) * Fix issue with remove-unnecessary-cast and necessary explicit casts. * Fix work item * Correct change ObjectCreationExpression to ImplictObjectCreationExpression for SmartBreakline (#60249) * Fix the initializer adding scenario * Fix the initializer removal scenario * Add an Assertion in debug to help if future derive type is added * Remove the check for NewKeyword * Option to always use default symbol servers for GTD (#60053) * pass the listener to the constructor * Edit the breaking change doc for publishing on docs (#60260) Co-authored-by: Rikki Gibson <rikkigibson@gmail.com> * Update SDK version in docs, etc. (#60299) * Fix type parameter scoping for local functions (#60098) Co-authored-by: Youssef1313 <youssefvictor00@gmail.com> * Fix conversion issues in null coalescing operator (#60202) * Revert "Fix a typo and an inconsistency" This reverts #60238 * Fix XLF files that were manually edited * Check for reference assembly before deciding which language to show (#60271) * Update dependencies from https://github.com/dotnet/arcade build 20220321.2 (#60315) [main] Update dependencies from dotnet/arcade * Fix issue when multiple different diagnostics share a single fixer * Add basic TS LSP server (#59639) * Add basic TS LSP server * Add external access layer for TS * Include text sync handlers * Fix build error * Switch to explicit interface impl * Fix namespaces * Remove testing string * Fix * Don't offer convert to switch when an implicit cast is present (#60224) * Initialize MEF parts before switching to the UI thread (#60289) * Add support for CheckedStatement for smart break line (#60308) * Add support for CheckedStatement * Address feedback * Remove a not needed import * Allow VS Mac to access a few options and a helper method (#60333) * [LSP] Don't adjust breakpoints when there are diagnostics in the document (#60334) * Fix our x86 test execution (#60323) * Fix our x86 test execution The transition to `dotnet test` seems to have broken our unit tests that depend on executing on an x86 architecture. This fixes that by using the `--arch` argument to the `dotnet test` execution. This was made more general to facilitate our future arm64 work in this area * Disable test failing on x64 * fix the test * Apply suggestions from code review Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com> * Disable test on spanish * Test fixes Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com> * Avoid resizing array builder while get unaliased references (#60220) This was being resized constantly after opening Orchard Core, allocating about 2 GB. The fact that is being called this many times appears to be another bug that I'm following up on. * Skip CodeDefinitionWindows integration tests * Fix top-level speculation (#60274) Co-authored-by: Youssef Victor <youssefvictor00@gmail.com> Co-authored-by: Sam Harwell <Sam.Harwell@microsoft.com> Co-authored-by: gel@microsoft.com <gel@microsoft.com> Co-authored-by: Gen Lu <genlu@users.noreply.github.com> Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com> Co-authored-by: Jonathon Marolf <jmarolf@users.noreply.github.com> Co-authored-by: Daniel Chalmers <daniel.chalmers@outlook.com> Co-authored-by: astroC86 <66444189+astroC86@users.noreply.github.com> Co-authored-by: David Wengier <david.wengier@microsoft.com> Co-authored-by: Cyrus Najmabadi <cyrusn@microsoft.com> Co-authored-by: Shen Chen <Cosifne@users.noreply.github.com> Co-authored-by: CyrusNajmabadi <cyrus.najmabadi@gmail.com> Co-authored-by: msftbot[bot] <48340428+msftbot[bot]@users.noreply.github.com> Co-authored-by: Bill Wagner <wiwagn@microsoft.com> Co-authored-by: Rikki Gibson <rikkigibson@gmail.com> Co-authored-by: Rikki Gibson <rigibson@microsoft.com> Co-authored-by: Julien Couvreur <jcouv@users.noreply.github.com> Co-authored-by: Youssef1313 <youssefvictor00@gmail.com> Co-authored-by: dotnet-maestro[bot] <42748379+dotnet-maestro[bot]@users.noreply.github.com> Co-authored-by: Joey Robichaud <jorobich@microsoft.com> Co-authored-by: David Barbet <dabarbet@microsoft.com> Co-authored-by: Jared Parsons <jaredpparsons@gmail.com> Co-authored-by: David Kean <davkean@microsoft.com>

* Fix our x86 test execution The transition to `dotnet test` seems to have broken our unit tests that depend on executing on an x86 architecture. This fixes that by using the `--arch` argument to the `dotnet test` execution. This was made more general to facilitate our future arm64 work in this area * Disable test failing on x64 * fix the test * Apply suggestions from code review Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com> * Disable test on spanish * Test fixes Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com>

jaredpar requested a review from a team as a code owner March 22, 2022 20:18

ghost added the Area-Infrastructure label Mar 22, 2022

333fred approved these changes Mar 22, 2022

View reviewed changes

JoeRobich approved these changes Mar 22, 2022

View reviewed changes

RikkiGibson approved these changes Mar 22, 2022

View reviewed changes

jaredpar enabled auto-merge (squash) March 22, 2022 20:28

runfoapp bot mentioned this pull request Mar 22, 2022

InitializeToolset flakiness in correctness builds #58199

Closed

Disable test failing on x64

6a36124

jaredpar requested a review from a team as a code owner March 22, 2022 22:20

fix the test

9e55412

sharwell approved these changes Mar 23, 2022

View reviewed changes

jaredpar commented Mar 23, 2022

View reviewed changes

src/EditorFeatures/CSharpTest2/EmbeddedLanguages/Json/CSharpJsonTests.cs Show resolved Hide resolved

jaredpar and others added 2 commits March 22, 2022 21:17

Apply suggestions from code review

ce00fb9

Co-authored-by: Sam Harwell <sam@tunnelvisionlabs.com>

Disable test on spanish

7de86c3

jaredpar requested a review from a team as a code owner March 23, 2022 15:27

michellemcdaniel mentioned this pull request Mar 23, 2022

Github experiencing degradation across all services dotnet/arcade#8666

Closed

jcouv reviewed Mar 23, 2022

View reviewed changes

jcouv approved these changes Mar 23, 2022

View reviewed changes

Test fixes

1cdb632

jaredpar disabled auto-merge March 23, 2022 22:08

jaredpar merged commit 8de5733 into dotnet:main Mar 23, 2022

jaredpar deleted the arch branch March 23, 2022 22:08

ghost added this to the Next milestone Mar 23, 2022

This was referenced Mar 24, 2022

csharpcodedefinitionwindow.codedefinitionwindowopensmetadataassource failing #60364

Closed

csharpcodedefinitionwindow.codedefinitionwindowopensmetadataassource failing #60363

Closed

333fred mentioned this pull request Mar 24, 2022

Merge main to features/required-members #60373

Merged

allisonchou modified the milestones: Next, 17.2.P3 Mar 28, 2022

Conversation

jaredpar commented Mar 22, 2022

Uh oh!

jaredpar commented Mar 22, 2022

Uh oh!

jaredpar commented Mar 22, 2022

Uh oh!

RikkiGibson commented Mar 22, 2022

Uh oh!

jaredpar commented Mar 22, 2022

Uh oh!

RikkiGibson commented Mar 22, 2022

Uh oh!

jaredpar commented Mar 22, 2022

Uh oh!

Uh oh!

sharwell Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jcouv Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

jaredpar Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

jcouv Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

jaredpar Mar 23, 2022

Choose a reason for hiding this comment

Uh oh!

jcouv commented Mar 23, 2022

Uh oh!

jcouv left a comment

Choose a reason for hiding this comment

Uh oh!

jaredpar commented Mar 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants