Skip to content

Conversation

@hebasto
Copy link
Member

@hebasto hebasto commented Jul 18, 2021

#20641 made building of the qt package in depends more structured, and, particularly, fixed cross-compiling detection on Intel-based macOS (x86_64).

However, a similar issue still present on M1-based macOS (arm64):

% make -C depends qt_configured
...
Configure summary:

Building on: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Building for: macx-clang (arm64, CPU features: neon crc32)
Target compiler: clang (Apple) 12.0.5
Configuration: cross_compile largefile neon precompile_header silent release c++11 c++14 c++1z reduce_exports static stl
...

Note an unexpected cross_compile.

% make -C depends qt_configured HOST=x86_64-apple-darwin18
...
Configure summary:

Build type: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Compiler: clang (Apple) 12.0.5
Configuration: sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl f16c largefile precompile_header rdrnd shani silent x86SimdAlways release c++11 c++14 c++1z reduce_exports static stl
...

Note that the build system failed to detect cross compiling.

In both cases building the configured qt package fails without installed Rosetta 2 (see #22402).

This PR:

With this PR:

% make -j 9 -C depends qt_configured
...
Configure summary:

Build type: macx-clang (arm64, CPU features: neon crc32)
Compiler: clang (Apple) 12.0.5
Configuration: largefile neon precompile_header silent release c++11 c++14 c++1z reduce_exports static stl
...
% make -j 9 -C depends qt_configured HOST=x86_64-apple-darwin18
...
Configure summary:

Building on: macx-clang (arm64, CPU features: neon crc32)
Building for: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Target compiler: clang (Apple) 12.0.5
Configuration: cross_compile sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl f16c largefile precompile_header rdrnd shani silent x86SimdAlways release c++11 c++14 c++1z reduce_exports static stl
...

This PR is based on #22506.

@hebasto
Copy link
Member Author

hebasto commented Jul 18, 2021

cc @promag @jarolrod @fanquake

@@ -0,0 +1,16 @@
To cross compile for the x86_64-darwin host on aarch64-darwin,
passing QMAKE_APPLE_DEVICE_ARCHS=arm64 to the configure script
does not work for some reasons.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we could document the reason it doesn't work. And any possible information on how/when/what circumstances this patch could be dropped

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer if we could document the reason it doesn't work.

I failed to track this bug down, unfortunately.

And any possible information on how/when/what circumstances this patch could be dropped

As this is an ad-hoc hack, we should try to drop it while bumping Qt version in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the #23583 is the correct solution for this problem.

Copy link
Contributor

@jarolrod jarolrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, thanks for fixing this. One concern: https://github.com/bitcoin/bitcoin/pull/22487/files#r671973286

Tested on an M1 Machine. Confirming that with this change we can drop the recommendation to install rosseta.

master:

a) native depends:

Configure summary:

Building on: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Building for: macx-clang (arm64, CPU features: neon crc32)
Target compiler: clang (Apple) 12.0.5
Configuration: cross_compile largefile neon precompile_header silent release c++11 c++14 c++1z reduce_exports static stl

b) building for x64:

Configure summary:

Build type: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Compiler: clang (Apple) 12.0.5
Configuration: sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl f16c largefile precompile_header rdrnd shani silent x86SimdAlways release c++11 c++14 c++1z reduce_exports static stl

pr:

a) native depends:

Configure summary:

Build type: macx-clang (arm64, CPU features: neon crc32)
Compiler: clang (Apple) 12.0.5
Configuration: largefile neon precompile_header silent release c++11 c++14 c++1z reduce_exports static stl

b) building for x64:

Configure summary:

Building on: macx-clang (arm64, CPU features: neon crc32)
Building for: macx-clang (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Target compiler: clang (Apple) 12.0.5
Configuration: cross_compile sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl f16c largefile precompile_header rdrnd shani silent x86SimdAlways release c++11 c++14 c++1z reduce_exports static stl

While here

I noticed the following line in the output when building qt through depends, how much of a concern can this be?:

> main.cpp:5:12: fatal error: 'linux/input.h' file not found
> #  include <linux/input.h>
>            ^~~~~~~~~~~~~~~
> 1 error generated.
> make[1]: *** [main.o] Error 1
test config.qtbase_gui.tests.evdev FAILED

@fanquake
Copy link
Member

fanquake commented Jul 19, 2021

I noticed the following line in the output when building qt through depends, how much of a concern can this be?:

It's not a concern. Output like that is expected while Qt is running it's feature checks.

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested ACK 5a55ba9.

However, I tried the following (hardcoded) and it worked fine:

$(package)_config_opts_darwin += -device-option QMAKE_APPLE_DEVICE_ARCHS=x86_64

make -C depends qt_configured HOST=x86_64-apple-darwin18
$(package)_config_opts_darwin += -device-option QMAKE_APPLE_DEVICE_ARCHS=arm64

make -C depends qt_configured

Note that it doesn't use _config_opts_x86_64_darwin or _config_opts_aarch64_darwin.

@hebasto
Copy link
Member Author

hebasto commented Jul 19, 2021

@promag

However, I tried the following (hardcoded) and it worked fine:

What is your Qt Configure summary?

UPDATE:

$(package)_config_opts_darwin += -device-option QMAKE_APPLE_DEVICE_ARCHS=arm64

make -C depends qt_configured

The -device-option will trigger cross-compiling. But a native compiling should be executed when not passing the HOST variable to the make -C depends.

Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does configuring with qt, for a x86-64 cross-compile, on an M1, using this PR, currently work?

@hebasto hebasto marked this pull request as draft July 20, 2021 10:03
@hebasto hebasto marked this pull request as ready for review July 20, 2021 15:20
@hebasto
Copy link
Member Author

hebasto commented Jul 20, 2021

Updated 5a55ba9 -> 8edac79 (pr22487.01 -> pr22487.02):

@hebasto
Copy link
Member Author

hebasto commented Jul 20, 2021

@fanquake

Does configuring with qt, for a x86-64 cross-compile, on an M1, using this PR, currently work?

Yes, it does, after the recent update.

@hebasto
Copy link
Member Author

hebasto commented Jul 23, 2021

Rebased 8edac79 -> d660e80 (pr22487.02 -> pr22487.03) due to the conflict with #22526.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 26, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #23580 (build: explicitly define previously implicit header include by kittywhiskers)
  • #23489 (build: Qt 5.15.2 by fanquake)
  • #22555 (build: Fix make apk for Android w/ non-default SOURCES_PATH in depends by hebasto)
  • #22552 (build: Improve depends build system robustness by hebasto)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@hebasto
Copy link
Member Author

hebasto commented Aug 3, 2021

Rebased d660e80 -> 85888b5 (pr22487.03 -> pr22487.04) due to the conflict with #22448.

@jarolrod
Copy link
Contributor

GUIX hashes

find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum

df863a06c4b6b708580cb82e807194793b9c3a8fb9396a66c0099c0c29617d6b  guix-build-85888b5210c4/output/aarch64-linux-gnu/SHA256SUMS.part
6e43e1faf23c13460a779aff681c0b39ab489cdc7ffc08656492638f0d1e9745  guix-build-85888b5210c4/output/aarch64-linux-gnu/bitcoin-85888b5210c4-aarch64-linux-gnu-debug.tar.gz
a23d0c0734f80e6faa15da864a71670131ab4ca0842a866585137f253d4aa6d7  guix-build-85888b5210c4/output/aarch64-linux-gnu/bitcoin-85888b5210c4-aarch64-linux-gnu.tar.gz
3d8f004f946919d0970ab7a01b8fc8d2708f67297e2e106009428b0512c13787  guix-build-85888b5210c4/output/arm-linux-gnueabihf/SHA256SUMS.part
bd6bc1c40cf664d0044ba21b4d026ba4b43d4b14b70552fc0ddbf3a93689c519  guix-build-85888b5210c4/output/arm-linux-gnueabihf/bitcoin-85888b5210c4-arm-linux-gnueabihf-debug.tar.gz
b0cb8eede06b51a359d7d8fed7adf4b1397d814b68d2e423470df62ba1fcca83  guix-build-85888b5210c4/output/arm-linux-gnueabihf/bitcoin-85888b5210c4-arm-linux-gnueabihf.tar.gz
b41950cc73a0130989c30093a27982ebfabb9f8de31f9310f06fa6e14644eeea  guix-build-85888b5210c4/output/dist-archive/bitcoin-85888b5210c4.tar.gz
57d07de9fe2e43a1b5cc508e13a517f7d224085f19cb169627b97d4e9ba61f18  guix-build-85888b5210c4/output/powerpc64-linux-gnu/SHA256SUMS.part
07f69a38755900293550aab3c603ed46193e6f3737e0dce3419bbe7e9d96a226  guix-build-85888b5210c4/output/powerpc64-linux-gnu/bitcoin-85888b5210c4-powerpc64-linux-gnu-debug.tar.gz
9469b2c83c5df17717ddab52015a352256fcf3241bd500321438558ceec8d21c  guix-build-85888b5210c4/output/powerpc64-linux-gnu/bitcoin-85888b5210c4-powerpc64-linux-gnu.tar.gz
57c2a2384e4e0feeaf5c261e0ba05652253e10bb84c1597e8b18c0b280ed0d6d  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/SHA256SUMS.part
a3587852be8dba91dac10a91f8827570677cae215afb61865dfa962b4d92044b  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/bitcoin-85888b5210c4-powerpc64le-linux-gnu-debug.tar.gz
b7a2b4e450a76bae79da8012b1b9ef70467d3760d61bc8a0d8b41271249caf44  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/bitcoin-85888b5210c4-powerpc64le-linux-gnu.tar.gz
cefaeb09ff6f6ed57850a1762090b58d118515d0e953d1d5eea1ad031f73dafa  guix-build-85888b5210c4/output/riscv64-linux-gnu/SHA256SUMS.part
b6897612ddca28534ffa61fd2bb321eafabcb01f3e16140934a732086930b851  guix-build-85888b5210c4/output/riscv64-linux-gnu/bitcoin-85888b5210c4-riscv64-linux-gnu-debug.tar.gz
90a4c28c2c85c562a0ba50cc158867d4634b6fa961fa61ec319fb87318f1780b  guix-build-85888b5210c4/output/riscv64-linux-gnu/bitcoin-85888b5210c4-riscv64-linux-gnu.tar.gz
ad48cd61446fbfc2e982aa2f6facced8b8c63c2606912fafea1eceadcf0ef1bd  guix-build-85888b5210c4/output/x86_64-apple-darwin18/SHA256SUMS.part
b3f866ed75e3e3eff5d6c565645abf7e6083de99c4c745d40c3053c60cebf7e5  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx-unsigned.dmg
c31f297161e002dd7e61bdc1ae511d6b39093c429cfe3b8f98ab51da3eca25eb  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx-unsigned.tar.gz
43835f47aeaa88c3b80b6623a9642f7ff6c887a41c30ab3c5f45ddafc004171b  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx64.tar.gz
f9a7c42bdf86aa1e16508f39f8144d3dd051a0a729fe3ffb6710e10723214ee5  guix-build-85888b5210c4/output/x86_64-linux-gnu/SHA256SUMS.part
da8b9e48cf9470e7105bd0614fefd618445cba9150a5ed3ca56b0a8f44133909  guix-build-85888b5210c4/output/x86_64-linux-gnu/bitcoin-85888b5210c4-x86_64-linux-gnu-debug.tar.gz
fbe71ec235394aed4643f5c48d4249a17b3769218757b5ac33c5a945d61c3236  guix-build-85888b5210c4/output/x86_64-linux-gnu/bitcoin-85888b5210c4-x86_64-linux-gnu.tar.gz
f10c3f187b9f19dcd0306e84bb81b06234bc48164377f9488ed457dfe024d438  guix-build-85888b5210c4/output/x86_64-w64-mingw32/SHA256SUMS.part
b59fed910a459911d7e088b5f9996ff2a5e271da1ccd3f20adc512fb64e73154  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win-unsigned.tar.gz
b5314730b0cf1176821bbc66f9c9ecec4e97bb189733533a80f043e0db49167f  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64-debug.zip
472b673f781d1dd0f411ef8ee42baf40e01592c82ec8538ab7df70c9758318c5  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64-setup-unsigned.exe
185abcc544ea7c625be27a866e804b23885e61abbb28937ed75976996fd39668  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64.zip

@hebasto
Copy link
Member Author

hebasto commented Aug 19, 2021

Guix builds:

$ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
df863a06c4b6b708580cb82e807194793b9c3a8fb9396a66c0099c0c29617d6b  guix-build-85888b5210c4/output/aarch64-linux-gnu/SHA256SUMS.part
6e43e1faf23c13460a779aff681c0b39ab489cdc7ffc08656492638f0d1e9745  guix-build-85888b5210c4/output/aarch64-linux-gnu/bitcoin-85888b5210c4-aarch64-linux-gnu-debug.tar.gz
a23d0c0734f80e6faa15da864a71670131ab4ca0842a866585137f253d4aa6d7  guix-build-85888b5210c4/output/aarch64-linux-gnu/bitcoin-85888b5210c4-aarch64-linux-gnu.tar.gz
3d8f004f946919d0970ab7a01b8fc8d2708f67297e2e106009428b0512c13787  guix-build-85888b5210c4/output/arm-linux-gnueabihf/SHA256SUMS.part
bd6bc1c40cf664d0044ba21b4d026ba4b43d4b14b70552fc0ddbf3a93689c519  guix-build-85888b5210c4/output/arm-linux-gnueabihf/bitcoin-85888b5210c4-arm-linux-gnueabihf-debug.tar.gz
b0cb8eede06b51a359d7d8fed7adf4b1397d814b68d2e423470df62ba1fcca83  guix-build-85888b5210c4/output/arm-linux-gnueabihf/bitcoin-85888b5210c4-arm-linux-gnueabihf.tar.gz
b41950cc73a0130989c30093a27982ebfabb9f8de31f9310f06fa6e14644eeea  guix-build-85888b5210c4/output/dist-archive/bitcoin-85888b5210c4.tar.gz
57d07de9fe2e43a1b5cc508e13a517f7d224085f19cb169627b97d4e9ba61f18  guix-build-85888b5210c4/output/powerpc64-linux-gnu/SHA256SUMS.part
07f69a38755900293550aab3c603ed46193e6f3737e0dce3419bbe7e9d96a226  guix-build-85888b5210c4/output/powerpc64-linux-gnu/bitcoin-85888b5210c4-powerpc64-linux-gnu-debug.tar.gz
9469b2c83c5df17717ddab52015a352256fcf3241bd500321438558ceec8d21c  guix-build-85888b5210c4/output/powerpc64-linux-gnu/bitcoin-85888b5210c4-powerpc64-linux-gnu.tar.gz
57c2a2384e4e0feeaf5c261e0ba05652253e10bb84c1597e8b18c0b280ed0d6d  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/SHA256SUMS.part
a3587852be8dba91dac10a91f8827570677cae215afb61865dfa962b4d92044b  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/bitcoin-85888b5210c4-powerpc64le-linux-gnu-debug.tar.gz
b7a2b4e450a76bae79da8012b1b9ef70467d3760d61bc8a0d8b41271249caf44  guix-build-85888b5210c4/output/powerpc64le-linux-gnu/bitcoin-85888b5210c4-powerpc64le-linux-gnu.tar.gz
cefaeb09ff6f6ed57850a1762090b58d118515d0e953d1d5eea1ad031f73dafa  guix-build-85888b5210c4/output/riscv64-linux-gnu/SHA256SUMS.part
b6897612ddca28534ffa61fd2bb321eafabcb01f3e16140934a732086930b851  guix-build-85888b5210c4/output/riscv64-linux-gnu/bitcoin-85888b5210c4-riscv64-linux-gnu-debug.tar.gz
90a4c28c2c85c562a0ba50cc158867d4634b6fa961fa61ec319fb87318f1780b  guix-build-85888b5210c4/output/riscv64-linux-gnu/bitcoin-85888b5210c4-riscv64-linux-gnu.tar.gz
ad48cd61446fbfc2e982aa2f6facced8b8c63c2606912fafea1eceadcf0ef1bd  guix-build-85888b5210c4/output/x86_64-apple-darwin18/SHA256SUMS.part
b3f866ed75e3e3eff5d6c565645abf7e6083de99c4c745d40c3053c60cebf7e5  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx-unsigned.dmg
c31f297161e002dd7e61bdc1ae511d6b39093c429cfe3b8f98ab51da3eca25eb  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx-unsigned.tar.gz
43835f47aeaa88c3b80b6623a9642f7ff6c887a41c30ab3c5f45ddafc004171b  guix-build-85888b5210c4/output/x86_64-apple-darwin18/bitcoin-85888b5210c4-osx64.tar.gz
f9a7c42bdf86aa1e16508f39f8144d3dd051a0a729fe3ffb6710e10723214ee5  guix-build-85888b5210c4/output/x86_64-linux-gnu/SHA256SUMS.part
da8b9e48cf9470e7105bd0614fefd618445cba9150a5ed3ca56b0a8f44133909  guix-build-85888b5210c4/output/x86_64-linux-gnu/bitcoin-85888b5210c4-x86_64-linux-gnu-debug.tar.gz
fbe71ec235394aed4643f5c48d4249a17b3769218757b5ac33c5a945d61c3236  guix-build-85888b5210c4/output/x86_64-linux-gnu/bitcoin-85888b5210c4-x86_64-linux-gnu.tar.gz
f10c3f187b9f19dcd0306e84bb81b06234bc48164377f9488ed457dfe024d438  guix-build-85888b5210c4/output/x86_64-w64-mingw32/SHA256SUMS.part
b59fed910a459911d7e088b5f9996ff2a5e271da1ccd3f20adc512fb64e73154  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win-unsigned.tar.gz
b5314730b0cf1176821bbc66f9c9ecec4e97bb189733533a80f043e0db49167f  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64-debug.zip
472b673f781d1dd0f411ef8ee42baf40e01592c82ec8538ab7df70c9758318c5  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64-setup-unsigned.exe
185abcc544ea7c625be27a866e804b23885e61abbb28937ed75976996fd39668  guix-build-85888b5210c4/output/x86_64-w64-mingw32/bitcoin-85888b5210c4-win64.zip

@fanquake
Copy link
Member

I'm closing this for now. In favour of #23583.

@fanquake fanquake closed this Nov 25, 2021
@hebasto hebasto deleted the 210718-m1qt branch November 26, 2021 12:57
@bitcoin bitcoin locked and limited conversation to collaborators Nov 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants