Skip to content

Conversation

@fanquake
Copy link
Member

How fast does this run? (-md).

@DrahtBot DrahtBot added the Tests label Sep 19, 2025
@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 19, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33436.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #33302 (ci: disable cirrus cache in 32bit arm job by willcl-ark)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@willcl-ark
Copy link
Member

I think this needs qemu:

commit 766b0b32361ae2eca3ee0b85caf1e27701fc0275
Author: will <will@256k1.dev>
Date:   Fri Sep 19 15:47:01 2025 +0100

    use qemu

diff --git a/.github/actions/configure-docker/action.yml b/.github/actions/configure-docker/action.yml
index c78df86b6cf..db5001c140d 100644
--- a/.github/actions/configure-docker/action.yml
+++ b/.github/actions/configure-docker/action.yml
@@ -4,9 +4,16 @@ inputs:
   use-cirrus:
     description: 'Use cirrus cache'
     required: true
+  use-qemu:
+    description: 'Use qemu'
+    default: false
 runs:
   using: 'composite'
   steps:
+    - name: Set up QEMU
+      if: ${{ inputs.use-qemu }}
+      uses: docker/setup-qemu-action@v3
+
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v3
       with:
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 6869c5cf25d..cf5ae7c717e 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -481,6 +481,7 @@ jobs:
             fallback-runner: 'ubuntu-24.04'
             timeout-minutes: 120
             file-env: './ci/test/00_setup_env_s390x.sh'
+            use-qemu: true
 
           - name: 'CentOS, depends, gui'
             cirrus-runner: 'ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-lg'
@@ -520,6 +521,7 @@ jobs:
         uses: ./.github/actions/configure-docker
         with:
           use-cirrus: ${{ needs.runners.outputs.use-cirrus-runners }}
+          use-qemu: ${{ matrix.use-qemu }}
 
       - name: Enable bpfcc script
         if: ${{ env.CONTAINER_NAME == 'ci_native_asan' }}

@willcl-ark
Copy link
Member

Or we just enable qemu on all jobs (no downsides that I know of) with:

commit 6e7fba0b93d1dec0deafe7b7f87054c9a629134f
Author: will <will@256k1.dev>
Date:   Fri Sep 19 15:47:01 2025 +0100

    use qemu

diff --git a/.github/actions/configure-docker/action.yml b/.github/actions/configure-docker/action.yml
index c78df86b6cf..b5f3601a24b 100644
--- a/.github/actions/configure-docker/action.yml
+++ b/.github/actions/configure-docker/action.yml
@@ -7,6 +7,9 @@ inputs:
 runs:
   using: 'composite'
   steps:
+    - name: Set up QEMU
+      uses: docker/setup-qemu-action@v3
+
     - name: Set up Docker Buildx
       uses: docker/setup-buildx-action@v3
       with:

which avoids passing options to the configure-docker action and things

@fanquake
Copy link
Member Author

Thanks @willcl-ark. Pulled in (some version) of your change, and added a NO_QT=1 for now.

@fanquake
Copy link
Member Author

Looks like this passed in 98m (without Qt and nothing cached).

@maflcko
Copy link
Member

maflcko commented Sep 23, 2025

Will the be useful overall? I think it is clear that no developer is using this platform right now (maybe not even a user), so the benefit of the CI config is mostly a sanity check. The issues it finds seems to happen less than once per year:

All the issues were trivial to fixup individually as a follow-up. However, the task here (even with a full depends cache) will take 60 minutes with an empty ccache (40 minutes compile + 20 minutes tests).

No strong opinion, but I think some tasks are fine as nightly tasks. It should be trivial to take the actions from this repo and call them from one of the nightly CI repos. C.f. https://github.com/bitcoin-core/qa-assets/blob/db30a6290f4070a31572801e2fd62c8876282ecc/.github/workflows/ci.yml#L56-L75

steps:
- name: Set up QEMU
if: ${{ inputs.use-qemu }}
uses: docker/setup-qemu-action@v3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this cache anything?

According to https://github.com/bitcoin/bitcoin/actions/caches

I see:

docker.io--tonistiigi--binfmt-latest-linux-x64 30 MB cached 4 days ago
refs/pull/33436/merge
Last used 4 days ago

Which seems a bit odd, but I am not familiar with GHA, nor the docker/ actions.

@willcl-ark
Copy link
Member

Will the be useful overall? I think it is clear that no developer is using this platform right now (maybe not even a user), so the benefit of the CI config is mostly a sanity check.

My understanding of this job is that it represents all Big Endian systems, and therefore, as you say just exists as a sanity check (that we don't have any endianness bugs). If nobody uses any BE systems (IBM Z, older SPARC, some powerpc) perhaps it's not even useful in that role though...

I'd agree with having this run as a nightly (or even less frequent) job. Would also note that secp256k1 are already running an s390x task:

https://github.com/bitcoin-core/secp256k1/blob/de6af6ae3568461edc9967500d7a70beeb66a642/.github/workflows/ci.yml#L161-L164

...so critical cryptography code is already being tested for endianness bugs that way. Perhaps that's enough testing for us?

@DrahtBot
Copy link
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

@l0rinc
Copy link
Contributor

l0rinc commented Sep 23, 2025

The issues it finds seems to happen less than once per year

For the record a big-endian CI machine would have helped with #31144

@maflcko
Copy link
Member

maflcko commented Sep 24, 2025

For the record a big-endian CI machine would have helped with #31144

It should be trivial to run the CI task locally, on demand. For development/debugging that should even be easier than to rely on a remove short-lived server. It should also be easy to temporarily add the config to the CI to trigger it on demand, if needed, similar to *BSD stuff, like #33435 (comment).

Again, I am not against adding this. My only concern is that this would likely be one of the slowest tasks. With the given timeout it doesn't even finish on GHA at all right now.

powerpc

I am sure there are some ppc users, so the s390x CI config must be kept and maintained and run before every release at least once. My only concern is that the overhead to run this on every pull request is more than the benefit it provides. (Similar to how there are no *BSD CI tasks on pull requests)

@fanquake
Copy link
Member Author

Agree that this specific job, isn't currently worth adding. Maybe we could add the cross-compile only, to match Guix.

@fanquake fanquake closed this Oct 15, 2025
@fanquake fanquake deleted the ci_run_s390x branch October 15, 2025 14:58
@hebasto
Copy link
Member

hebasto commented Oct 16, 2025

Maybe we could add the cross-compile only, to match Guix.

What does it mean? We do not build for the s390x-linux-gnu host in the Guix scripts?

@maflcko
Copy link
Member

maflcko commented Oct 16, 2025

I'd expect that cross-compilation in this CI task always succeeds, if all the other CI tasks (including the other cross-compile ones). For s390x the value really is in running the full tests to see any endian bugs.

@maflcko
Copy link
Member

maflcko commented Dec 17, 2025

Maybe by cross-compilation and only running the tests on s390x, it could be speed up?

@l0rinc
Copy link
Contributor

l0rinc commented Dec 17, 2025

That sounds like a good idea, that's the most painful part.
Is cross compilation stable cross-endianness? I guess it has to be, has anyone tried it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants