Internal Code Merge and update branding to 5.0.3 by Anipik · Pull Request #46888 · dotnet/runtime

Anipik · 2021-01-12T23:06:35Z

Internal Code Merge
Update Branding to 5.0.3
Update PackageIndex
Remove building of shipped packages

ghost · 2021-01-12T23:06:41Z

Tagging subscribers to this area: @safern, @ViktorHofer
See info in area-owners.md if you want to be subscribed.

Issue Details

Internal Code Merge
Update Branding to 5.0.3
Update PackageIndex
Remove building of shipped packages

Author:	Anipik
Assignees:	-
Labels:	`area-Infrastructure-libraries`
Milestone:	-

eng/common/post-build/publish-using-darc.ps1

Anipik · 2021-01-13T02:20:03Z

@akoeplinger can you take a look at the mono failure here ?

akoeplinger · 2021-01-13T17:07:26Z

@Anipik hmm there's no Mono-specifics at that point, it should be identical to CoreCLR. I do see that the projects it complains about missing are indeed disabled on Mono:

/Users/runner/work/1/s/src/coreclr/tests/src/Directory.Build.targets(153,6): error : The native project files are missing in /Users/runner/work/1/s/artifacts/tests/coreclr/obj/OSX.x64.Release/Native/Interop/ICustomMarshaler/ConflictingNames/ please run build from the root of the repo at least once [/Users/runner/work/1/s/src/tests/Interop/ICustomMarshaler/ConflictingNames/SameNameDifferentAssembly.csproj]
/Users/runner/work/1/s/src/coreclr/tests/src/Directory.Build.targets(153,6): error : The native project files are missing in /Users/runner/work/1/s/artifacts/tests/coreclr/obj/OSX.x64.Release/Native/Interop/ICustomMarshaler/ConflictingNames/ please run build from the root of the repo at least once [/Users/runner/work/1/s/src/tests/Interop/ICustomMarshaler/ConflictingNames/MultipleALCs.csproj]
/Users/runner/work/1/s/src/coreclr/tests/src/Directory.Build.targets(153,6): error : The native project files are missing in /Users/runner/work/1/s/artifacts/tests/coreclr/obj/OSX.x64.Release/Native/Interop/LayoutClass/ please run build from the root of the repo at least once [/Users/runner/work/1/s/src/tests/Interop/LayoutClass/LayoutClassTest.csproj]
/Users/runner/work/1/s/src/coreclr/tests/src/Directory.Build.targets(153,6): error : The native project files are missing in /Users/runner/work/1/s/artifacts/tests/coreclr/obj/OSX.x64.Release/Native/Interop/PInvoke/DateTime/ please run build from the root of the repo at least once [/Users/runner/work/1/s/src/tests/Interop/PInvoke/DateTime/DateTimeTest.csproj]

Not sure why/how that could be triggered due to this PR though.

Anipik · 2021-01-13T18:17:44Z

@akoeplinger any suggestions on fixing this ?

danmoseley · 2021-01-13T19:54:21Z

Raw log from the Unzip native test artifacts step just before this failing Copy native test components to test output folder step

https://dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_apis/build/builds/947215/logs/1954

In there I see .dylib files being extracted for other test libraries, but not for these four.

Now if I look in the CoreCLR native test builds, I see them being built for Debug but not for Release
https://dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_apis/build/builds/947215/logs/829
https://dev.azure.com/dnceng/9ee6d478-d288-47f7-aacc-f6e6d082ae6d/_apis/build/builds/947215/logs/528

danmoseley · 2021-01-13T19:57:11Z

The release build seems to have failed here (silently)

2021-01-12T23:29:37.5425270Z [ 97%] Linking CXX shared library libDllImportPath_Local.dylib
2021-01-12T23:29:37.5530700Z [ 97%] Linking CXX shared library libAnsiBStrTestNative.dylib
2021-01-12T23:29:37.6099410Z /bin/sh: /bin/sh: cannot execute binary file
2021-01-12T23:29:37.6201080Z make[1]: *** [Interop/StringMarshalling/AnsiBSTR/CMakeFiles/AnsiBStrTestNative.dir/all] Error 126
2021-01-12T23:29:37.6301230Z make[1]: *** Waiting for unfinished jobs....

whereas on debug it did not.

danmoseley · 2021-01-13T20:01:54Z

It seems this error can mean that the system is asked to execute a binary that was built for a different architecture, but perhaps that's not the only possibility.
Adding @jkoritzinsky -- any reason why linking this test folder failed this way?

More (?) importantly, why did the build step not fail as a whole? See it failed even though make returned error 2:

2021-01-12T23:29:37.5319300Z [ 97%] Linking CXX shared library libExecInDefAppDomDll.dylib
2021-01-12T23:29:37.5425270Z [ 97%] Linking CXX shared library libDllImportPath_Local.dylib
2021-01-12T23:29:37.5530700Z [ 97%] Linking CXX shared library libAnsiBStrTestNative.dylib
2021-01-12T23:29:37.6099410Z /bin/sh: /bin/sh: cannot execute binary file
2021-01-12T23:29:37.6201080Z make[1]: *** [Interop/StringMarshalling/AnsiBSTR/CMakeFiles/AnsiBStrTestNative.dir/all] Error 126
2021-01-12T23:29:37.6301230Z make[1]: *** Waiting for unfinished jobs....
2021-01-12T23:29:37.6401920Z [ 97%] Built target ExecInDefAppDomDll
2021-01-12T23:29:37.6478070Z [ 97%] Built target DllImportPath_Local
2021-01-12T23:29:37.6579990Z [ 98%] Linking CXX shared library libExactSpellingNative.dylib
2021-01-12T23:29:37.6974160Z [ 98%] Built target ExactSpellingNative
2021-01-12T23:29:37.6981190Z make: *** [all] Error 2
2021-01-12T23:29:37.7002170Z Test build successful.
2021-01-12T23:29:37.7009820Z Test binaries are available at /Users/runner/work/1/s/artifacts/tests/coreclr/OSX.x64.Release

https://dev.azure.com/dnceng/public/_build/results?buildId=947215&view=logs&jobId=d9e9ad30-bad9-5a6a-fbc8-d1db16d2799e&j=f2a23cee-3431-50cb-c21a-1d5929055501&t=f6769e10-5d04-5826-7c9b-eeb756bab9ac

danmoseley · 2021-01-13T20:03:20Z

I do not know how to continue investigation as I'm not familiar with how cmake works/our makefiles.

akoeplinger · 2021-01-13T20:17:05Z

Hmm yeah the build step not failing even though there's an error is concerning.

safern · 2021-01-13T21:59:15Z

It seems like somehow we're loosing the last exit code:

runtime/src/coreclr/build-test.sh

Line 747 in 17fc554

if [[ "$?" -ne 0 ]]; then

danmoseley · 2021-01-13T22:02:18Z

If we had an idea how to generate that message, probably you could easily track down where the error code went. But I do not - if the linker itself was corrupted, I would imagine the other libraries would fail to build.

is there anything special these 4 have in common? or perhaps there is a race condition that only affected them - something getting overwritten?

safern · 2021-01-13T22:08:51Z

Actually this is where we lost the exit code:

runtime/src/coreclr/build-test.sh

Line 374 in 17fc554

if [[ "$?" -ne 0 ]]; then

And that's why build_Tests didn't exit with exit code 1.

I can see this executed because of this message print in the logs:

runtime/eng/native/build-commons.sh

Line 78 in 17fc554

    
           echo "Commencing build of \"$message\" for $__TargetOS.$__BuildArch.$__BuildType in $intermediatesDir"

So it seems like build_native didn't return a failing exit code here:

runtime/eng/native/build-commons.sh

Line 197 in 17fc554

exit "$exit_code"

danmoseley · 2021-01-13T22:25:10Z

Which presumably means that cmake --build "/Users/runner/work/1/s/artifacts/tests/coreclr/obj/OSX.x64.Release/Native" --target install -- -j 4 did not return an error code for some reason.

Question remains what 2021-01-12T23:29:37.6099410Z /bin/sh: /bin/sh: cannot execute binary file is coming from...

danmoseley · 2021-01-13T22:27:41Z

Might it be interesting to pull down the payload (that the test step is extracting from) -- it contains the cmake files for each library?

safern · 2021-01-13T22:38:12Z

In that case I think @trylek or @jkoritzinsky could point us towards the right direction on how to try and repro this with what we have from the build to figure out what's swallowing the exit code

trylek · 2021-01-13T23:18:40Z

runtime/eng/native/build-commons.sh

Line 187 in 17fc554

$cmake_command --build "$intermediatesDir" --target install -- -j "$__NumProc"

seems to be missing the error check bit we now have in the equivalent place

runtime/eng/native/build-commons.sh

Line 200 in 6ded57b

exit_code="$?"

danmoseley · 2021-01-13T23:42:26Z

Aha - yes, that's necessary because the next line CFLAGS="${SAVED_CFLAGS}" resets the error code!

dan@LAPTOP-P6UJDVTA:~$ echo $?
127
dan@LAPTOP-P6UJDVTA:~$ FOO=BAR
dan@LAPTOP-P6UJDVTA:~$ echo $?
0

This is not the case on Windows.

Any way we could search for any other such places to fix?

safern · 2021-01-13T23:51:19Z

Ah right. Thanks @trylek I totally missed that setting vars resets the exit code 😄

Anipik · 2021-01-13T23:53:02Z

would it make sense to merge this branding PR and create an issue to track this failure ? this will help us make progress on merging the servicing approved changes.

safern · 2021-01-13T23:56:35Z

Yeah, however it would be good to actually add this:

runtime/eng/native/build-commons.sh

Line 200 in 6ded57b

exit_code="$?"

to 5.0 branch.

trylek · 2021-01-14T00:03:50Z

For "other places to fix", we have some that Jarret collected based on developer reports in

#44393

I'm not sure if I see any easy way to auto-scan the scripts for swallowed error codes like this.

danmoseley · 2021-01-14T00:14:02Z

@Anipik the change to cherry-pick is cd7d23f

Anipik · 2021-01-14T00:15:32Z

@Anipik the change to cherry-pick is cd7d23f

i will cherry-pick and push

When native build fails, handle exit code properly by checking it immediately after cmake is called

danmoseley · 2021-01-14T02:54:07Z

The only failure is this on Mono:
System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest8_Values

probably not worth logging/investigating unless it reoccurs, since this is a servicing branch.


Error message
System.Threading.Tasks.TaskSchedulerException : An exception was thrown by a TaskScheduler.\n---- System.ExecutionEngineException : Couldn't create thread. Error 0x0


Stack trace
   at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:line 1696
   at System.Threading.Tasks.Task`1[[System.Int32, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].StartNew(Task parent, Func`1 function, CancellationToken cancellationToken, TaskCreationOptions creationOptions, InternalTaskOptions internalOptions, TaskScheduler scheduler) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:line 333
   at System.Threading.Tasks.TaskFactory.StartNew[Int32](Func`1 function, CancellationToken cancellationToken, TaskCreationOptions creationOptions, TaskScheduler scheduler) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskFactory.cs:line 629
   at System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest8_Values() in /_/src/libraries/System.Threading/tests/ThreadLocalTests.cs:line 253
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) in /_/src/mono/netcore/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.cs:line 384
----- Inner Stack Trace -----
   at System.Threading.Thread.Start(Object parameter) in /_/src/mono/netcore/System.Private.CoreLib/src/System/Threading/Thread.Mono.cs:line 265
   at System.Threading.Tasks.ThreadPoolTaskScheduler.QueueTask(Task task) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/ThreadPoolTaskScheduler.cs:line 50
   at System.Threading.Tasks.TaskScheduler.InternalQueueTask(Task task) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/TaskScheduler.cs:line 237
   at System.Threading.Tasks.Task.ScheduleAndStart(Boolean needsProtection) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:line 1668

danmoseley · 2021-01-14T02:58:18Z

@Anipik mergeable?

Anipik · 2021-01-14T03:24:34Z

@Anipik mergeable?

yes we can merge this one. waiting for a review cc @safern @ViktorHofer @ericstj @aik-jahoda for a review ?

mmitche and others added 12 commits December 10, 2020 10:12

Set publishing to temporarily use internal/release-publishing branch

681e2c3

Merge in 'release/5.0' changes

aeea2ad

Merge in 'release/5.0' changes

56ff2f6

Merge in 'release/5.0' changes

4f8dd29

Merge in 'release/5.0' changes

728fdef

Merge in 'release/5.0' changes

0260401

Merge in 'release/5.0' changes

968afdf

Merge in 'release/5.0' changes

2d2f618

Merge in 'release/5.0' changes

db62a11

Merge in 'release/5.0' changes

1cd455b

Merge in 'release/5.0' changes

cb5f173

update branding to 5.0.3

7d67ace

ghost added the area-Infrastructure-libraries label Jan 12, 2021

Anipik requested review from ViktorHofer, aik-jahoda and safern January 12, 2021 23:06

safern reviewed Jan 12, 2021

View reviewed changes

eng/common/post-build/publish-using-darc.ps1 Outdated Show resolved Hide resolved

Anipik mentioned this pull request Jan 13, 2021

[release/5.0] Fix EventLog on Windows ARM64 #46598

Merged

Anipik mentioned this pull request Jan 13, 2021

[release/5.0] Backport System.Speech to compat pack #46729

Merged

runfoapp bot mentioned this pull request Jan 13, 2021

System.Threading.Tests.ThreadLocalTests.RunThreadLocalTest8_Values failing on Mono on SLES 12 AMD64 #43981

Open

revert the branch name to master

f2d9d80

Fix Unix build issue (#42104)

18bc2c4

When native build fails, handle exit code properly by checking it immediately after cmake is called

safern approved these changes Jan 14, 2021

View reviewed changes

Anipik merged commit 9874123 into dotnet:release/5.0 Jan 14, 2021

Anipik deleted the branding503 branch January 14, 2021 03:34

ghost locked as resolved and limited conversation to collaborators Feb 13, 2021

Conversation

Anipik commented Jan 12, 2021

Uh oh!

ghost commented Jan 12, 2021

Uh oh!

Uh oh!

Anipik commented Jan 13, 2021

Uh oh!

akoeplinger commented Jan 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Anipik commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

akoeplinger commented Jan 13, 2021

Uh oh!

safern commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

safern commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

safern commented Jan 13, 2021

Uh oh!

trylek commented Jan 13, 2021

Uh oh!

danmoseley commented Jan 13, 2021

Uh oh!

safern commented Jan 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Anipik commented Jan 13, 2021

Uh oh!

safern commented Jan 13, 2021

Uh oh!

trylek commented Jan 14, 2021

Uh oh!

danmoseley commented Jan 14, 2021

Uh oh!

Anipik commented Jan 14, 2021

Uh oh!

danmoseley commented Jan 14, 2021

Uh oh!

danmoseley commented Jan 14, 2021

Uh oh!

Anipik commented Jan 14, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

akoeplinger commented Jan 13, 2021 •

edited

Loading

danmoseley commented Jan 13, 2021 •

edited

Loading

safern commented Jan 13, 2021 •

edited

Loading