tools: switch LibreSSL to OpenSSL 1.1.1 by commodo · Pull Request #4749 · openwrt/openwrt

commodo · 2021-11-05T10:41:25Z

There may be a growing trend to start migrating back to OpenSSL, since it
has recovered quite a bit from the HeartBleed incident.
In the meantime, LibreSSL has been lagging behind in some features.
And a number of applications are trying to make use of these new features
and algorithms of OpenSSL.

Some links:
https://lwn.net/Articles/841664/
https://www.python.org/dev/peps/pep-0644/#compatibility
https://www.python.org/dev/peps/pep-0644/#libressl-support

In my case, Python (starting with version 3.10) has started to enforce
OpenSSL 1.1.1 as a minimum. This makes it difficult for the python3 package
(the host version) to support HTTP SSL requests.
The SSL module of the host Python3 is required to download packages from
pypi.org for some Python3 packages in the packages feed.

An alternative would be to just patch CPython to work with libressl,
however this may be an early sign that other tools (CMake comes to mind)
would also start preferring OpenSSL.

LibreSSL has been forked from 1.0.1g in 2014.

While both SSL implementations have significant contributions, there seems
to be more push (newer features and contributors) on OpenSSL than LibreSSL.

There's even an OpenSSL 3.0.0 version. That version may be too new for
what's needed right now, but 1.1.1 seems like a good version to use right
now in the tools/ folder.

Signed-off-by: Alexandru Ardelean ardeleanalex@gmail.com

cd3f6ee build, cmake: add quotes for FW_UTIL variable arguments f4d1263 build, cmake: switch OPENSSL_CRYPTO_LIBRARY -> OPENSSL_CRYPTO_LIBRARIES Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

This is a rough adaptation from tools/libressl with some bits from the package/libs/openssl build rules. There may be a growing trend to start migrating back to OpenSSL, since it has recovered quite a bit from the HeartBleed incident. In the meantime, LibreSSL has been lagging behind in some features. And a number of applications are trying to make use of these new features and algorithms of OpenSSL. Some links: https://lwn.net/Articles/841664/ https://www.python.org/dev/peps/pep-0644/#compatibility https://www.python.org/dev/peps/pep-0644/#libressl-support In my case, Python (starting with version 3.10) has started to enforce OpenSSL 1.1.1 as a minimum. This makes it difficult for the python3 package (the host version) to support HTTP SSL requests. The SSL module of the host Python3 is required to download packages from pypi.org for some Python3 packages in the packages feed. An alternative would be to just patch CPython to work with libressl, however this may be an early sign that other tools (CMake comes to mind) would also start preferring OpenSSL. LibreSSL has been forked from 1.0.1g in 2014. While both SSL implementations have significant contributions, there seems to be more push (newer features and contributors) on OpenSSL than LibreSSL. There's even an OpenSSL 3.0.0 version. That version may be too new for what's needed right now, but 1.1.1 seems like a good version to use right now in the tools/ folder. For this build (particularly), the OpenSSL build disables shared-libs to make sure only static libs are provided. HW and assembly stuff is disabled. And thread support is enabled. This may be disable-able, but pthread support should be common in most places OpenWrt gets built. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

Now with OpenSSL 1.1.1 present, we can use it in the tools folder and remove the LibreSSL variant. It's usually too much hassle to keep both around. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

Since LibreSSL has been switched in favor of OpenSSL, this patch can be dropped. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

commodo · 2021-11-05T10:42:20Z

ping @neheb
you seemed a bit interested in this OpenSSL 1.1.1 thing;
maybe ping other interested parties?

neheb · 2021-11-05T10:53:28Z

I prefer keeping LibreSSL. The build system is a lot more sane.

commodo · 2021-11-05T12:00:35Z

I prefer keeping LibreSSL. The build system is a lot more sane.

yeah, OpenSSL's build system is a bit specific/quirky;
well, i will try to think of a way that works for Python;
keeping LibreSSL and adding a host-build for OpenSSL would be a bit nuts as well;
the headers and libs would overlap each other;
or, we could add a separate prefix for OpenSSL in staging_dir, but ¯_(ツ)_/¯ ;
that's also just polluting the staging dir

i'll keep this PR here for a few days to sort of raise attention to it;
then i'll just keep the branch in my tree;

in the meantime I think I can try to patch Python to work with LibreSSL;
hopefully that will not evolve into something really annoying to keep patching;

i do feel that the idea to switch tools/libressl -> tools/openssl may come back later (not necessarily from my side);

ptpt52 · 2021-11-06T00:17:09Z

I prefer the openssl

dhewg · 2021-11-06T06:24:42Z

Is that an actual problem you ran into, as in you updated the python3 package to 3.10?
I didn't dig into this at all, but it seems somewhat unlikely, as that would prevent *BSD to update their python, no?

commodo · 2021-11-08T07:23:42Z

Is that an actual problem you ran into, as in you updated the python3 package to 3.10? I didn't dig into this at all, but it seems somewhat unlikely, as that would prevent *BSD to update their python, no?

yes; it actually fails to build the _ssl module in the host Python build;
some functions are missing in OpenBSD's LibreSSL;
and we need it for the Python build to download stuff from pypi.org

i did not check whether they updated their Python version;
but I did see that they did update some things in their libssl folder, to add some of the the missing functions that would help Python work with _ssl;
but that also means waiting for them to release a newer (standalone) LibreSSL [i'd be fine with that];

right now, I'm also working on a patch to support _ssl on host-Python with LibreSSL;
it's compiling, but not yet working;
patch looks like this (it gives an idea of what's missing in LibreSSL 3.4.1);

--- a/Modules/_ssl.c
+++ b/Modules/_ssl.c
@@ -67,7 +67,47 @@
 #  error "OPENSSL_THREADS is not defined, Python requires thread-safe OpenSSL"
 #endif
 
+#ifdef OPENWRT_HOST_BUILD
+static int SSL_write_ex(SSL *s, const void *buf, size_t num, size_t *written)
+{
+    int ret = SSL_write(s, buf, num);
+    if (ret > 0)
+        *written = ret;
+    if (ret < 0)
+        ret = 0;
+    return (ret > 0);
+}
 
+static int SSL_read_ex(SSL *s, void *buf, size_t num, size_t *readbytes)
+{
+    int ret = SSL_read(s, buf, num);
+    if (ret > 0)
+        *readbytes = ret;
+    if (ret < 0)
+        ret = 0;
+    return (ret > 0);
+}
+
+static STACK_OF(X509) *SSL_get0_verified_chain(const SSL *s)
+{
+    return NULL;
+}
+
+static int SSL_CTX_set_num_tickets(SSL_CTX *ctx, size_t num_tickets)
+{
+    return 0;
+}
+
+size_t SSL_CTX_get_num_tickets(const SSL_CTX *ctx)
+{
+    return 0;
+}
+
+static int SSL_CTX_get_security_level(const SSL_CTX *ctx)
+{
+    return 0;
+}
+#endif
 
 struct py_ssl_error_code {
     const char *mnemonic;
--- a/Modules/_ssl/debughelpers.c
+++ b/Modules/_ssl/debughelpers.c
@@ -7,6 +7,13 @@
 #define SSL3_MT_CHANGE_CIPHER_SPEC              0x0101
 #endif
 
+#ifdef OPENWRT_HOST_BUILD
+typedef void (*SSL_CTX_keylog_cb_func)(const SSL *ssl, const char *line);
+static void SSL_CTX_set_keylog_callback(SSL_CTX *ctx, SSL_CTX_keylog_cb_func cb)
+{
+}
+#endif
+
 static void
 _PySSL_msg_callback(int write_p, int version, int content_type,
                     const void *buf, size_t len, SSL *ssl, void *arg)

dhewg · 2021-11-08T07:37:15Z

Huh, okay, that indeed looks problematic, thanks!

I don't know how urgent it is to bump the python3 package to >=3.10, but maybe wait a little 'till the dust settles.
It doesn't sound like support for libressl is gone for good: python/cpython#28728 (review)
Maybe it all just works out with either of the parties releasing point releases to improve the situation.

commodo · 2021-11-08T07:52:20Z

Huh, okay, that indeed looks problematic, thanks!

I don't know how urgent it is to bump the python3 package to >=3.10, but maybe wait a little 'till the dust settles. It doesn't sound like support for libressl is gone for good: python/cpython#28728 (review) Maybe it all just works out with either of the parties releasing point releases to improve the situation.

Well, it's not hurry from my side to update.
Though, this update is a little rockier than others (due to this OpenSSL story).
I've been working on this update on-n-off for 1-2 weeks now, trying to balance things.
Maybe this will settle in the next 2-4 weeks (which is fine).
I'll try to do a patch for CPython with LibreSSL, mostly to remove this from my work-queue for now.

In recent years, I also started trying to address some things at CPython level.
Like these here.
I got a bit of push-back from CPython on this OpenSSL think (see python/cpython#29237 & https://bugs.python.org/issue45627 )
There's also hashlib that doesn't build without OpenSSL 1.1.1, but that lib isn't important for the host-side Python.

Yeah, navigating people's preferences is what this seems to boil down to :)

dhewg · 2021-11-08T08:00:43Z

Comparing OpenBSD's commit history with the diff you provided:
openbsd/src@b5b7f1d
openbsd/src@4a18b5b
openbsd/src@8095b13
openbsd/src@d1ec7c3

If your diff is complete (in the sense of python >= 3.10 requirements) it appears only SSL_CTX_get_security_level is missing atm. So yes, waiting for a release including those symbols is most likely way easier that pushing for s/libressl/openssl/ :)

commodo · 2021-11-08T08:04:28Z

If your diff is complete (in the sense of python >= 3.10 requirements) it appears only SSL_CTX_get_security_level is missing atm. So yes, waiting for a release including those symbols is most likely way easier that pushing for s/libressl/openssl/ :)

yeah, i found the same things last week;
but i only started looking into their tree after doing this PR;
the push-back I got from CPython is why I initially went for this PR/change

dhewg · 2021-11-08T08:22:20Z

OpenBSD's ports has python 3.10, and with its system libressl it seems they only need this one patch for SSL_CTX_get_security_level: https://github.com/openbsd/ports/blob/master/lang/python/3.10/patches/patch-Modules__ssl_c

commodo · 2021-11-08T08:51:39Z

OpenBSD's ports has python 3.10, and with its system libressl it seems they only need this one patch for SSL_CTX_get_security_level: https://github.com/openbsd/ports/blob/master/lang/python/3.10/patches/patch-Modules__ssl_c

Thanks :)
I was thinking about looking for it later.
So, this means, we wait for their LibreSSL release and use this patch.

I'll close this PR in the next few days.

dhewg · 2021-11-08T12:00:09Z

Looks like that's indeed going to be easier now, but maybe there're other arguments for OpenSSL?
Dunno, I just got a little bit confused as it almost sounded like *BSD are stuck with <3.10 heh :)

commodo · 2021-11-08T12:12:14Z

Looks like that's indeed going to be easier now, but maybe there're other arguments for OpenSSL? Dunno, I just got a little bit confused as it almost sounded like *BSD are stuck with <3.10 heh :)

Well, this also influenced me a bit: https://lwn.net/Articles/841664/
It's a not very strong against LibreSSL, but it looks like some frustration venting.
There's also some contribution numbers comparisons there (per LibreSSL and per OpenSSL).

The Python core dev (who handles SSL) also pitched in there with a comment :

Posted Jan 4, 2021 23:21 UTC (Mon) by tiran (guest, #94212) [Link]
As the maintainer of CPython's ssl module I agree with the article.\
A short while ago I have created a Python enhancement 
proposal to drop compatibility with old OpenSSL versions and require 
OpenSSL >= 1.1.1 in Python 3.10. The same PEP 
also proposes to remove LibreSSL support from upstream CPython. 
Python core development doesn't have resources to support LibreSSL.
It's taking considerable time to work around API incompatibilities. 
Missing APIs are also blocking new features.

https://www.python.org/dev/peps/pep-0644/#libressl-support

And it does indeed feel that LibreSSL has been playing catch-up with OpenSSL.
For me, it's just Python 3.10 I care about.
Let's see how this evolves later :)

dhewg · 2021-11-09T09:20:04Z

I can't really comment on all the details, but the frustrations seem to go both ways:
https://gitlab.alpinelinux.org/alpine/tsc/-/issues/28

commodo · 2021-11-10T07:25:02Z

I can't really comment on all the details, but the frustrations seem to go both ways: https://gitlab.alpinelinux.org/alpine/tsc/-/issues/28

figures i guess :)
well, we'll see ¯_(ツ)_/¯

dhewg · 2021-11-10T09:09:31Z

@commodo see ^
That may influence how you wanna proceed.
Maybe reopen this or backport the libressl patches for the next half year?

commodo · 2021-11-10T11:40:28Z

@commodo see ^ That may influence how you wanna proceed. Maybe reopen this or backport the libressl patches for the next half year?

oh; thanks for pitching in that discussion :)
so, backporting patches is the path of least resistance for now;

TBH, I don't care which {Libre,Open}SSL we use, as long as I don't need to manually spin-up patches which are also hard to sell to other people in the community.
If it's about Python, it's fine; I co-maintain it, so it's easier.

For now, switching LibreSSL to OpenSSL is a hard-sell; and I suspected it wouldn't fly.
But at least we got here, which mostly raises awareness on this.

dhewg · 2021-11-10T17:21:49Z

Sounds like a plan, I guess I would do the same (for the record: I don't care that much either, I just believe that changing the host SSL lib is more painful).

Just so I get the full picture (I don't use any python packages on-target):
If a python target package is build with/using pypi, that drags in the host python package, which then depends on the host SSL lib to download stuff over https? Without pypi there wouldn't be any need for the host python package?

commodo · 2021-11-11T08:53:43Z

Sounds like a plan, I guess I would do the same (for the record: I don't care that much either, I just believe that changing the host SSL lib is more painful).

I wouldn't change the tools/ssl lib either.
But, I don't know how painful it is to switch either.

Just so I get the full picture (I don't use any python packages on-target): If a python target package is build with/using pypi, that drags in the host python package, which then depends on the host SSL lib to download stuff over https? Without pypi there wouldn't be any need for the host python package?

Python-target always requires Python-host.
It's one of the [since-forever] quirks of cross-building Python for a specific platform.
At some stage in the build, the same host-Python interpreter is required to do some stuff (which I forget) for the target-Python.
When building natively, the build re-uses the interpreter from the same build.
If the versions wouldn't need to match so closely, we wouldn't need to build the host-Python.

For other Python packages for target-build, the host-Python should have the _ssl module available, so that pip can download from https:// URLs (from pypi.org usually).
So, Python (itself) will build fine without this SSL stuff, but other Python packages (for target and host builds) won't download; which will fill-up the buildbot faillogs.

commodo · 2021-11-12T11:22:45Z

urgs....
even with LibreSSL's patches, I still get:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError('No cipher can be selected.'))': /simple/ply/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError('No cipher can be selected.'))': /simple/ply/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError('No cipher can be selected.'))': /simple/ply/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError('No cipher can be selected.'))': /simple/ply/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError('No cipher can be selected.'))': /simple/ply/
Could not fetch URL https://pypi.org/simple/ply/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/ply/ (Caused by SSLError(SSLError('No cipher can be selected.'))) - skipping

oh well, back to the drawing board;
i guess i may not patch LibreSSL at all;
it looks like I will just need to patch Python myself in a special way;
but i still felt like venting about this a bit;
and it's likely that OpenBSD's Python suffers from a similar issue, but they don't know it [maybe]

dhewg · 2021-11-12T15:13:15Z

Ugh, maybe some defines within python need to be adapted?
But don't worry, we've all been had these annoying bang-head-on-keyboad problems :)

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

commodo · 2021-11-14T04:18:25Z

for posterity;
i've cracked it: openwrt/packages#17121

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

dhewg · 2021-11-16T08:51:36Z

Alright, so I assume patching libressl wasn't successful?

commodo · 2021-11-16T09:51:47Z

Alright, so I assume patching libressl wasn't successful?

It may work to patch libressl, but I'd avoid it.
I have more control over what goes into Python than LibreSSL and OpenWrt core :)

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Add patch: 029-disable-deprecation-warning.patch Fixes apparmor build. The warning causes a configure error. Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Add patch: 029-disable-deprecation-warning.patch Fixes apparmor build. The warning causes a configure error. Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com> Signed-off-by: Tianling Shen <cnsztl@immortalwrt.org>

Manually re-applied: 008-distutils-use-python-sysroot.patch 016-adjust-config-paths.patch Drop patch: 003-do-not-run-distutils-tests.patch There is now a configure option '--disable-test-modules' And seems we left the '_ctypes_test' around for quite some time. Dropped now. Refs: https://bugs.python.org/issue27640 https://bugs.python.org/issue43282 Drop patch: 013-getbuildinfo-date-time-source-date-epoch.patch Python build honors SOURCE_DATE_EPOCH pretty well now. Drop setuptools patches. Setuptools should be reproducible with Python 3.6+ according to a mention here: pypa/setuptools#1690 (comment) It's time to let upstream fix Setuptools reproduce-ability. Drop patch: 010-do-not-add-rt-lib-dirs-when-cross-compiling.patch I can't seem to fully remember why it's there. And it seem to build fine without it. Drop patch: 015-abort-on-failed-modules.patch Python build supports a similar PYTHONSTRICTEXTENSIONBUILD=1 env-var option. Add patch: 026-openssl-feature-flags.patch We need to keep this in our tree for a while. See: https://bugs.python.org/issue45627 Backport patch: 027-bpo-43158-Use-configure-values-for-building-_uuid-ex.patch Link: python/cpython#29353 Fixes the build for uuid C module. Add patch: 028-host-python-support-ssl-with-libressl.patch We need the _ssl module working on the host-side with LibreSSL for pip to work to download from https://pypi.org Refs: openwrt/openwrt#4749 Add patch: 029-disable-deprecation-warning.patch Fixes apparmor build. The warning causes a configure error. Refreshed the rest of patches. Some old build-flags were removed. They don't seem to be necessary anymore. Split python3-uuid from python3-light. To better manage the libuuid library (if needed). Also, fixing the uuid C module build. Seems this was failing, and was falling back to using hashlib. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

commodo added 4 commits November 5, 2021 12:14

firmware-utils: bump to latest git HEAD

4e27671

cd3f6ee build, cmake: add quotes for FW_UTIL variable arguments f4d1263 build, cmake: switch OPENSSL_CRYPTO_LIBRARY -> OPENSSL_CRYPTO_LIBRARIES Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

tools: libressl: drop package and switch to openssl

7c61b10

Now with OpenSSL 1.1.1 present, we can use it in the tools folder and remove the LibreSSL variant. It's usually too much hassle to keep both around. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

tools: cmake: remove libressl patch-fix

3366e41

Since LibreSSL has been switched in favor of OpenSSL, this patch can be dropped. Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>

mans0n added the build/scripts/tools pull request/issues for build, scripts and tools related changes label Nov 5, 2021

commodo closed this Nov 10, 2021

dhewg mentioned this pull request Nov 10, 2021

Challenges and future issues libressl/portable#617

Open

commodo mentioned this pull request Nov 14, 2021

python3: bump to version 3.10.0 openwrt/packages#17121

Merged

dhewg mentioned this pull request Jul 18, 2022

openssl: add host build recipe #10299

Open

commodo mentioned this pull request Jul 20, 2022

python3: update patches for libressl to fix hashlib build error openwrt/packages#18981

Merged

commodo deleted the openwrt-openssl-1.1.1 branch April 8, 2023 12:07

Uh oh!

Conversation

commodo commented Nov 5, 2021

Uh oh!

commodo commented Nov 5, 2021

Uh oh!

neheb commented Nov 5, 2021

Uh oh!

commodo commented Nov 5, 2021

Uh oh!

ptpt52 commented Nov 6, 2021

Uh oh!

dhewg commented Nov 6, 2021

Uh oh!

commodo commented Nov 8, 2021

Uh oh!

dhewg commented Nov 8, 2021

Uh oh!

commodo commented Nov 8, 2021

Uh oh!

dhewg commented Nov 8, 2021

Uh oh!

commodo commented Nov 8, 2021

Uh oh!

dhewg commented Nov 8, 2021

Uh oh!

commodo commented Nov 8, 2021

Uh oh!

dhewg commented Nov 8, 2021

Uh oh!

commodo commented Nov 8, 2021

Uh oh!

dhewg commented Nov 9, 2021

Uh oh!

commodo commented Nov 10, 2021

Uh oh!

dhewg commented Nov 10, 2021

Uh oh!

commodo commented Nov 10, 2021

Uh oh!

dhewg commented Nov 10, 2021

Uh oh!

commodo commented Nov 11, 2021

Uh oh!

commodo commented Nov 12, 2021

Uh oh!

dhewg commented Nov 12, 2021

Uh oh!

commodo commented Nov 14, 2021

Uh oh!

dhewg commented Nov 16, 2021

Uh oh!

commodo commented Nov 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants