Improve the Windows OneCore target support by MouriNaruto · Pull Request #7729 · openssl/openssl

MouriNaruto · 2018-11-29T07:35:17Z

Add New Targets
- VC-WIN32-UWP
- VC-WIN64A-UWP
- VC-WIN32-ARM-UWP
- VC-WIN64-ARM-UWP
How to compile OpenSSL libraries for an Universal Windows Platform (UWP) App

perl Configure --release no-asm no-hw no-dso no-tests no-stdio no-async VC-WIN64A-UWP
nmake build_libs

It is my first time to make a pull request to OpenSSL. So I need some help.
Thanks.

Mouri_Naruto

CLA: trivial

…ct-Nagisa/openssl-Microsoft into OpenSSL_1_1_1_WinRT-stable

Configurations/50-win-onecore.conf

lindexi · 2018-11-29T08:13:14Z

OpenSSL for Windows Universal API (OneCore)

compile openssl (version >= 1.1.0) for Universal Windows Platform ARM architecture

#4853

#4156

MouriNaruto · 2018-12-03T06:10:55Z

@dot-asm I need your help.

dot-asm · 2018-12-04T11:14:56Z

@dot-asm I need your help.

This is misdirected question, address the project.

MouriNaruto · 2018-12-05T07:16:55Z

@dot-asm I have renamed the name of pull request because you say it is a misdirected question. Or I created a wrong pull request? How to do in the next?

levitte · 2018-12-05T10:01:20Z

What @dot-asm means it that he doesn't want to be addressed directly, that you should address the project, i.e. @openssl.

I've decided to take this on, but then I need your help to understand what you're trying to achieve. I have no idea what WindowsApp.lib, and trying to search for it gets me to pages talking about OneCore libs... It would be great if the comments, either in commit messages or in the conf file you've added to, expanded a bit on what's going on, and what that covers that isn't already covered by the current onecore targets we already have.

MouriNaruto · 2018-12-05T11:13:36Z

@levitte Thanks, I want to say sorry to @dot-asm after I read your reply. (I don't know what he said before because I am not good at English. I am really sorry. 😓)

what you're trying to achieve

I want to make the OpenSSL can build the static and dynamic libraries which the UWP store app can use it. In OpenSSL 1.0.2 era, I commit the similar thing to Microsoft's OpenSSL repo before. But when I found many people have done something in the offical OpenSSL repo, I want to add them to the offical instead of the Microsoft's because it can help more UWP app developers.

what WindowsApp.lib

The WindowsApp.lib is the contains all Win32 APIs which the UWP store app can use. If you want to build an UWP app which can upload to the Microsoft Store, you only can use the Win32 APIs defined in this lib.

Based on the Microsoft's branch, I only add some configurations to Configurations/50-win-onecore.conf and ms/uwp_applink.h.

I add the ms/uwp_applink.h because we can't use GetModuleHandle API in the store app, I use the macro to hack it like the Microsoft's way.
The Microsoft's way is:
https://github.com/Microsoft/openssl/blob/a033d72bdb8cc465594a8061eea369cacc6b7027/ms/winrt.cpp#L153
https://github.com/Microsoft/openssl/blob/a033d72bdb8cc465594a8061eea369cacc6b7027/ms/winrtdef.h#L49

More questions are welcome. Thank you.

Mouri.

MouriNaruto · 2018-12-19T23:58:03Z

@levitte Any news?

levitte · 2018-12-20T08:23:17Z

Sorry, been distracted by other stuff... I'll run some tests now

levitte · 2018-12-20T10:58:14Z

So I got quite a bit, but here's a showstopper I can't figure out:

        link /nologo /debug /dll /NODEFAULTLIB:kernel32.lib /APPCONTAINER /nologo /debug  /implib:libcrypto.lib /out:libcrypto-1_1-x64.dll /def:libcrypto.def @C:\Users\Richard\AppData\Local\Temp\nm90E.tmp || (DEL /Q libcrypto-1_1-x64.* libcrypto.lib && EXIT 1)
   Creating library libcrypto.lib and object libcrypto.exp
MSVCRT.lib(gs_report.obj) : error LNK2019: unresolved external symbol __imp_RtlCaptureContext referenced in function capture_current_context
MSVCRT.lib(utility_desktop.obj) : error LNK2001: unresolved external symbol __imp_RtlCaptureContext
MSVCRT.lib(gs_report.obj) : error LNK2019: unresolved external symbol __imp_RtlVirtualUnwind referenced in function capture_current_context
MSVCRT.lib(utility_desktop.obj) : error LNK2001: unresolved external symbol __imp_RtlVirtualUnwind
MSVCRT.lib(gs_report.obj) : error LNK2019: unresolved external symbol __imp_UnhandledExceptionFilter referenced in function __raise_securityfailure
MSVCRT.lib(utility_desktop.obj) : error LNK2001: unresolved external symbol __imp_UnhandledExceptionFilter
MSVCRT.lib(utility_desktop.obj) : error LNK2019: unresolved external symbol __imp_GetStartupInfoW referenced in function __scrt_get_show_window_mode
MSVCRT.lib(utility_desktop.obj) : error LNK2019: unresolved external symbol __imp_GetModuleHandleW referenced in function __scrt_is_managed_app
libcrypto-1_1-x64.dll : fatal error LNK1120: 5 unresolved externals
NMAKE : fatal error U1077: 'link' : return code '0x1'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\amd64\nmake.exe"' : return code '0x2'
Stop.

levitte · 2018-12-20T11:01:08Z

BTW, I did a few changes in addition to your stuff:

diff --git a/Configurations/10-main.conf b/Configurations/10-main.conf
index 5cf345da0..e98ea6f92 100644
--- a/Configurations/10-main.conf
+++ b/Configurations/10-main.conf
@@ -1290,7 +1290,7 @@ my %targets = (
     },
     "VC-WIN64I" => {
         inherit_from     => [ "VC-WIN64-common", asm("ia64_asm"),
-                              sub { $disabled{shared} ? () : "ia64_uplink" } ],+                              sub { $disabled{uplink} ? () : "ia64_uplink" } ],         AS               => "ias",
         ASFLAGS          => "-d debug",
         asoutflag        => "-o ",
@@ -1302,7 +1302,7 @@ my %targets = (
     },
     "VC-WIN64A" => {
         inherit_from     => [ "VC-WIN64-common", asm("x86_64_asm"),
-                              sub { $disabled{shared} ? () : "x86_64_uplink" } ],
+                              sub { $disabled{uplink} ? () : "x86_64_uplink" } ],
         AS               => sub { vc_win64a_info()->{AS} },
         ASFLAGS          => sub { vc_win64a_info()->{ASFLAGS} },
         asoutflag        => sub { vc_win64a_info()->{asoutflag} },
@@ -1315,7 +1315,7 @@ my %targets = (
     },
     "VC-WIN32" => {
         inherit_from     => [ "VC-noCE-common", asm("x86_asm"),
-                              sub { $disabled{shared} ? () : "uplink_common" } ],
+                              sub { $disabled{uplink} ? () : "uplink_common" } ],
         CFLAGS           => add("/WX"),
         AS               => sub { vc_win32_info()->{AS} },
         ASFLAGS          => sub { vc_win32_info()->{ASFLAGS} },
@@ -1378,7 +1378,7 @@ my %targets = (
 #### MinGW
     "mingw" => {
         inherit_from     => [ "BASE_unix", asm("x86_asm"),
-                              sub { $disabled{shared} ? () : "x86_uplink" } ],
+                              sub { $disabled{uplink} ? () : "x86_uplink" } ],
         CC               => "gcc",
         CFLAGS           => picker(default => "-Wall",
                                    debug   => "-g -O0",
diff --git a/Configurations/50-win-onecore.conf b/Configurations/50-win-onecore.conf
index b6291a558..606be55db 100644
--- a/Configurations/50-win-onecore.conf
+++ b/Configurations/50-win-onecore.conf
@@ -71,6 +71,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
      "VC-WIN64A-UWP" => {
@@ -80,6 +81,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
     "VC-WIN32-ARM-UWP" => {
@@ -89,6 +91,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
      "VC-WIN64-ARM-UWP" => {
@@ -98,6 +101,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
 );
d+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
      "VC-WIN64A-UWP" => {
@@ -80,6 +81,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
     "VC-WIN32-ARM-UWP" => {
@@ -89,6 +91,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
      "VC-WIN64-ARM-UWP" => {
@@ -98,6 +101,7 @@ my %targets = (
         lflags          => add("/APPCONTAINER"),
         defines         => add("WINAPI_FAMILY=WINAPI_FAMILY_APP",
                                "_WIN32_WINNT=0x0A00"),
+        disable         => [ 'dso', 'ui-console', 'stdio', 'async' ],
         ex_libs         => "WindowsApp.lib",
     },
 );
diff --git a/Configure b/Configure
index d5dc36c28..f9cf4f524 100755
--- a/Configure
+++ b/Configure
@@ -409,6 +409,7 @@ my @disablables = (
     "ubsan",
     "ui-console",
     "unit-test",
+    "uplink",
     "whirlpool",
     "weak-ssl-ciphers",
     "zlib",
@@ -482,7 +483,7 @@ my @disable_cascades = (

     # Without position independent code, there can be no shared libraries or DSOs
     "pic"               => [ "shared" ],
-    "shared"            => [ "dynamic-engine" ],
+    "shared"            => [ "dynamic-engine", "uplink" ],
     "engine"            => [ "afalgeng", "devcryptoeng" ],

     # no-autoalginit is only useful when building non-shared

levitte · 2018-12-20T11:01:50Z

With that change, I configured like this:

perl Configure VC-WIN64A-UWP no-asm no-uplink

MouriNaruto · 2018-12-20T12:27:48Z

@levitte It's excellent! Thanks. But I have some questions.

Why don't add the "no-hw" option?
What's the role of "uplink" in OpenSSL project? Why can you disable it directly instead of making a "macro hack" like the Microsoft's OpenSSL fork?
I don't know whether MSVC toolchain support the OpenSSL's assembler code. So is it suitable for us not to include the "no-asm" option in the configuration?

I also hope you will forgive my ignorance, thanks.

Mouri.

levitte · 2018-12-20T12:34:15Z

I'm not sure no-hw is necessary
uplink implements the library side of what's described in https://www.openssl.org/docs/manmaster/man3/OPENSSL_Applink.html
If you look at my patch above, you'll notice that I added "uplink" among things the can be disabled. That's how I avoid the macro hack.
The toolchain can handle it, but it seems that some assembler modules depend on stuff that isn't available with WindowsApp.lib, so it seems they need to be turned off, at least until we figure out how they can be used.

levitte · 2018-12-20T12:37:16Z

Note btw that I'm on vacation starting tomorrow and will be away for two weeks. In all likelyhood, this won't be done today, so we will just have to continue after I get back

MouriNaruto · 2018-12-20T12:57:26Z

@levitte Got it. Have a nice vocation!

I'm not sure no-hw is necessary

It may be necessary.

I read the "https://wiki.openssl.org/index.php/Compilation_and_Installation" and have found "no-hw | Disables hardware support (useful on mobile devices)". You may know the developer can make their UWP apps running on the mobile phones. And I also read the compile scripts in the Microsoft's OpenSSL branch and I found they have used the "no-hw" options. (https://github.com/Microsoft/openssl/blob/a033d72bdb8cc465594a8061eea369cacc6b7027/ms/do_vsprojects15.bat#L53)

Mouri

levitte · 2018-12-20T14:14:29Z

Adding no-hw to my configuration line changes nothing at all

MouriNaruto · 2019-01-12T14:06:23Z

@levitte Any news?

MouriNaruto · 2019-02-11T08:02:59Z

@levitte How's it going?

levitte · 2019-02-11T10:20:13Z

no-hw isn't really of much value today, and terribly misunderstood... it was originally conceived to disable building of any ENGINE module that supported HSMs (the first ENGINE module was to support the nCipher device, which needs support files, such as nCipher specific header files, libraries, ...). Now that there are a number of ENGINE modules that are not about supporting specific hardware, no-hw has become a bit of an oddity.

levitte · 2019-02-11T10:25:49Z

Could it be that with no-hw, you really want to disable all ENGINE module building (as well as the ENGINE API)? In that case, no-engine would make more sense, wouldn't you think?

MouriNaruto · 2019-02-11T12:01:42Z

@levitte Yes, I don't want to disable all ENGINE modules. But can this project compile successfully without this option? I don‘t know.

In short, it will be fine if it compiles successfully without it.

Finally, I would like to know if OpenSSL will determine the hardware instruction support at run time.

Thank you.

Mouri

levitte · 2019-02-11T12:21:15Z

Finally, I would like to know if OpenSSL will determine the hardware instruction support at run time.

Yup, and that's completely unrelated to no-hw, I might add. There's the internal function OPENSSL_cpuid_setup that deals with that kind of thing.

MouriNaruto · 2019-02-11T12:36:26Z

@levitte Got it. It's your turn to do now. (Thank you very much.)

Mouri.

MouriNaruto · 2019-02-11T16:38:55Z

I forgot to ask, which thing I should do which can help you next? Thanks.

@levitte

MouriNaruto · 2019-03-30T15:59:01Z

I close the PR because it is not responding. If someone want to merge them, tell me to reopen. Thanks.

MouriNaruto · 2019-03-30T16:05:45Z

I reopen the PR because I still want to make this PR to be merged.

But the process of OpenSSL Team is too slow to wait. (sad)

Mouri

MouriNaruto · 2019-03-30T16:10:56Z

#7729 (comment)

Where I can find your modify branch? @levitte

levitte · 2019-04-01T04:58:50Z

Pardon, I've been distracted by a gazillion other OpenSSL stuff...

illera88 · 2019-04-22T22:32:36Z

Hi @levitte how is this going? Fixing this is blocking fixing microsoft/vcpkg#4267.

I would appreciate your help.

thank you in advance

illera88 · 2019-05-06T19:01:01Z

@levitte sorry to bother you again about this. Do you have plans to work on this?

MouriNaruto · 2019-05-11T21:22:56Z

Closed because of #8917.

Add the Universal Windows Platform (UWP) App Support.

292e53f

openssl-machine added the hold: cla required The contributor needs to submit a license agreement label Nov 29, 2018

MouriNaruto added 2 commits November 29, 2018 15:55

Add the Universal Windows Platform (UWP) App Support.

20bcbc1

CLA: trivial

Merge branch 'OpenSSL_1_1_1_WinRT-stable' of https://github.com/Proje…

ee29a76

…ct-Nagisa/openssl-Microsoft into OpenSSL_1_1_1_WinRT-stable

openssl-machine removed the hold: cla required The contributor needs to submit a license agreement label Nov 29, 2018

lindexi reviewed Nov 29, 2018

View reviewed changes

Configurations/50-win-onecore.conf Outdated Show resolved Hide resolved

Fix

98c1e28

MouriNaruto mentioned this pull request Nov 29, 2018

OpenSSL for Windows Universal API (OneCore) #4853

Closed

MouriNaruto changed the title ~~Add the Universal Windows Platform (UWP) App Support.~~ Improve the Windows OneCore target support Dec 4, 2018

levitte self-assigned this Dec 5, 2018

MouriNaruto mentioned this pull request Mar 7, 2019

OpenSSL 1.1.1 (LTS) microsoft/openssl#65

Closed

MouriNaruto closed this Mar 30, 2019

MouriNaruto reopened this Mar 30, 2019

MouriNaruto mentioned this pull request Apr 9, 2019

Do we have plan to create openssl 1.0.2r branch for UWP platform? microsoft/openssl#71

Closed

MouriNaruto mentioned this pull request May 11, 2019

Add targets in Windows OneCore target for making building libraries for Windows Store apps easier. #8917

Closed

MouriNaruto closed this May 11, 2019

Uh oh!

Conversation

MouriNaruto commented Nov 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

lindexi commented Nov 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MouriNaruto commented Dec 3, 2018

Uh oh!

dot-asm commented Dec 4, 2018

Uh oh!

MouriNaruto commented Dec 5, 2018

Uh oh!

levitte commented Dec 5, 2018

Uh oh!

MouriNaruto commented Dec 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

levitte commented Dec 5, 2018

Uh oh!

MouriNaruto commented Dec 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MouriNaruto commented Dec 19, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

MouriNaruto commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

MouriNaruto commented Dec 20, 2018

Uh oh!

levitte commented Dec 20, 2018

Uh oh!

MouriNaruto commented Jan 12, 2019

Uh oh!

MouriNaruto commented Feb 11, 2019

Uh oh!

levitte commented Feb 11, 2019

Uh oh!

levitte commented Feb 11, 2019

Uh oh!

MouriNaruto commented Feb 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

levitte commented Feb 11, 2019

Uh oh!

MouriNaruto commented Feb 11, 2019

Uh oh!

MouriNaruto commented Feb 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MouriNaruto commented Mar 30, 2019

Uh oh!

MouriNaruto commented Mar 30, 2019

Uh oh!

MouriNaruto commented Mar 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

levitte commented Apr 1, 2019

Uh oh!

illera88 commented Apr 22, 2019

Uh oh!

illera88 commented May 6, 2019

Uh oh!

MouriNaruto commented May 11, 2019

Uh oh!

Reviewers

Assignees

MouriNaruto commented Nov 29, 2018 •

edited

Loading

lindexi commented Nov 29, 2018 •

edited

Loading

MouriNaruto commented Dec 5, 2018 •

edited

Loading

MouriNaruto commented Dec 5, 2018 •

edited

Loading

MouriNaruto commented Feb 11, 2019 •

edited

Loading

MouriNaruto commented Feb 11, 2019 •

edited

Loading

MouriNaruto commented Mar 30, 2019 •

edited

Loading