Risc-V builders fail to wget (large files?)

Bug #2109593 reported by Nathan Teodosio
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Snapcraft
Confirmed
Critical
Unassigned

Bug Description

Trying to build llvm-chromium for Risc-V, two builds failed at the same spot:

--->
Pulling llvm
[28/Apr/2025:21:06:47 +0000] "CONNECT commondatastorage.googleapis.com:443 HTTP/1.1" 200 1486418453 "-" "-"
None
Detailed information: ChunkedEncodingError
<---

Where the llvm part's source is

  https://commondatastorage.googleapis.com/chromium-browser-official/chromium-135.0.7023.0.tar.xz

Full log: https://people.canonical.com/~npt/buildlog_snap_ubuntu_noble_riscv64_llvm-chromium_BUILDING.txt.gz
Snapcraft.yaml: https://git.launchpad.net/~chromium-team/+git/llvm-chromium/tree/snapcraft.yaml#n24

Revision history for this message
Jürgen Gmach (jugmac00) wrote :

Thanks for the report.

Pulling 6.5 GiB might be the reason, but there is no obvious timeout we are running into.

Quentin, who is on support next week, will look into this.

Have you ever built this successfully before on Launchpad?

Revision history for this message
Nathan Teodosio (nteodosio) wrote : Re: [Bug 2109593] Re: Risc-V builders fail to wget (large files?)

No.

By the way, I tried the same wget in a Risc-V virtual machine just to be sure,
and I can confirm that it had no problem completing.

Revision history for this message
Tushar Gupta (tushar5526) wrote :

Hi, thanks for reporting. Did this started to fail recently, or has this always been the case?

It would be quite easy to investigate if you can set up the snap build on https://qastaging.launchpad.net/ where we can easily access the builder and investigate the issue easily.

Changed in launchpad:
status: New → Triaged
importance: Undecided → Critical
Revision history for this message
Nathan Teodosio (nteodosio) wrote :

> Hi, thanks for reporting. Did this started to fail recently, or has this
> always been the case?

I never before tried a Risc-V Chromium build.

> It would be quite easy to investigate if you can set up the snap build
> on https://qastaging.launchpad.net/ where we can easily access the
> builder and investigate the issue easily.

https://qastaging.launchpad.net/ does not have RiscV in the list of options:

Processors: (Optional)
 AMD x86-64 (amd64)
 ARM ARMv8 (arm64)
 ARM ARMv7 Hard Float (armhf)
 Intel x86 (i386)
 PowerPC64 Little-Endian (ppc64el)
 IBM System z (s390x)

description: updated
description: updated
Revision history for this message
Enrique Sánchez (enriqueesanchz) wrote (last edit ):

`snapcraft pull` downloads the entire file in about 30 mins (and the proxy will be idle since then) and then proceeds to copy it to a specific folder. During this copy the proxy timeouts and kills all processes.

Made a workaround using `nil` plugin and `wget` in the `override-build` phase in `snapcraft.yaml`: https://git.launchpad.net/~enriqueesanchz/+git/llvm-chromiummm-enrique/tree/snapcraft.yaml

It takes only 30 minutes to download using `wget` and ~1 hour to unpackage it while using `snapcraft pull` increasing the timeout took ~5-6 hours to finish.

Revision history for this message
Enrique Sánchez (enriqueesanchz) wrote :

I've filed this bug to snapcraft: https://github.com/canonical/snapcraft/issues/5611

Changed in launchpad:
status: Triaged → In Progress
affects: launchpad → snapcraft
Changed in snapcraft:
status: In Progress → Confirmed
Revision history for this message
Alex Lowe (lengau) wrote :

Thanks for the snapcraft bug! I've put some details that should help us determine the best way to fix the speed issue there.

As a note on the workaround in snapcraft.yaml: the following `override-pull` is faster than the `override-build` addition and might in fact be what you want to do regardless so you don't have to write the entire chromium tarball to disk:

override-pull: |
  wget -nv -O - https://commondatastorage.googleapis.com/chromium-browser-official/chromium-135.0.7023.0.tar.xz \
   | xz -d -T0 \
   | tar --strip-components 1 --wildcards -x -C $CRAFT_PART_SRC chromium-*/tools/clang chromium-*/third_party/llvm*

This drops pull time for me on a Star64 board from 36 minutes to 13 minutes by:

1. Extracting whilst downloading
2. Using multi-threaded xz decompression
3. Only extracting the necessary directories

In fact, doing this on my (amd64) laptop significantly sped up the pull as well, taking 3m17s compared to the default behaviour's 30m+ whilst still building a correct snap.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.