This issue is essentially an expanded follow-up to #14340.
I've found the following packages from Mantevo that produce checksum failures and cannot be extracted from their tarballs during install:
cloverleaf
minimd
minighost
minismac2d
minixyce
hpccg
pathfinder
tealeaf
There are several things that these packages have in common:
- All come from this GitHub repo.
- The URL that Spack is using to get the tarball points to a Git LFS pointer file.
- The actual file being obtained with
curl seems to be the Git LFS pointer file. However, I'm still not positive that this is the case.
- The pointer file's
oid field matches the SHA256 checksum that is in the Spack package file.
- The checksum that causes the mismatch can be reproduced by running
sha256sum on this pointer file.
Additionally, when I tried manually extracting the actual tarball for cloverleaf using curl on the command line (following these instructions), I got the following:
$ curl -X POST -d '{"operation":"download", "objects":[{"oid": "sha256:de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f ", "size": 1756656}]}' https://github.com/Mantevo/mantevo.github.io.git/info/lfs/objects/batch
{
"message": "This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.",
"documentation_url": "https://help.github.com/articles/purchasing-additional-storage-and-bandwidth-for-an-organization/"
}
To keep this issue from getting too lengthy, I'm only going to include the output from trying to install cloverleaf below, but I have gotten the same errors for all of the above packages.
Spack version
$ spack --version
# On my personal laptop (Ubuntu 18.04 x86_64)
0.14.0-24-ae8782852
# On UTK's Power9 Cluster
0.14.0-21-676eb56
Steps to reproduce the issue
# For the checksum error
$ spack install cloverleaf
# For the tarball extraction error
$ spack install -n cloverleaf
Errors
On my personal laptop:
$ spack install cloverleaf
==> 13573: Installing cloverleaf
==> Fetching from /home/ilumsden/spack/var/spack/cache/_source-cache/archive/de/de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f.tar.gz failed.
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Error: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
==> Error: Failed to install cloverleaf due to ChildError: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1152, in do_stage
self.do_fetch(mirror_only)
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1132, in do_fetch
self.stage.check()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 490, in check
self.fetcher.check()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 492, in check
"Expected %s but got %s" % (self.digest, checker.sum))
spack.fetch_strategy.ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
$ spack install -n cloverleaf
==> 13599: Installing cloverleaf
==> Fetching from /home/ilumsden/spack/var/spack/cache/_source-cache/archive/de/de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f.tar.gz failed.
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Staging archive: /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz
/bin/tar: This does not look like a tar archive
gzip: stdin: not in gzip format
/bin/tar: Child returned status 1
/bin/tar: Error is not recoverable: exiting now
==> Error: ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'
==> Error: Failed to install cloverleaf due to ChildError: ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1153, in do_stage
self.stage.expand_archive()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 528, in expand_archive
self.fetcher.expand()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 434, in expand
decompress(self.archive_file)
File "/home/ilumsden/spack/lib/spack/spack/util/executable.py", line 189, in __call__
proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 2:
'/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-agtpltse7hfn4qefyuwruo4lwn7eueoe/CloverLeaf-1.1.tar.gz'
On UTK's Power9 Cluster:
$ spack install cloverleaf
==> 94167: Installing cloverleaf
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Error: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
==> Error: Failed to install cloverleaf due to ChildError: ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
/home/ilumsden/spack/lib/spack/spack/package.py:1131, in do_fetch:
1128 self.stage.fetch(mirror_only)
1129 self._fetch_time = time.time() - start_time
1130
>> 1131 if checksum and self.version in self.versions:
1132 self.stage.check()
1133
1134 self.stage.cache_local()
Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1152, in do_stage
self.do_fetch(mirror_only)
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1132, in do_fetch
self.stage.check()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 490, in check
self.fetcher.check()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 492, in check
"Expected %s but got %s" % (self.digest, checker.sum))
spack.fetch_strategy.ChecksumError: sha256 checksum failed for /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
Expected de87f7ee6b917e6b3d243ccbbe620370c62df890e3ef7bdbab46569b57be132f but got 5a69bc3d1b4ca0d071609ecd844ba46636d5239a0acfa56e3c93fa606e112ab9
$ spack install -n cloverleaf
==> 98004: Installing cloverleaf
==> Fetching https://github.com/Mantevo/mantevo.github.io/raw/master/download_files/CloverLeaf-1.1.tar.gz
################################################################################################################################## 100.0%################################################################################################################################## 100.0%
==> Staging archive: /tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz
/usr/bin/tar: This does not look like a tar archive
gzip: stdin: not in gzip format
/usr/bin/tar: Child returned status 1
/usr/bin/tar: Error is not recoverable: exiting now
==> Error: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'
==> Error: Failed to install cloverleaf due to ChildError: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'Traceback (most recent call last):
File "/home/ilumsden/spack/lib/spack/spack/build_environment.py", line 801, in child_process
return_value = function()
File "/home/ilumsden/spack/lib/spack/spack/installer.py", line 984, in build_process
pkg.do_patch()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1167, in do_patch
self.do_stage()
File "/home/ilumsden/spack/lib/spack/spack/package.py", line 1153, in do_stage
self.stage.expand_archive()
File "/home/ilumsden/spack/lib/spack/spack/util/pattern.py", line 68, in getter
getattr(item, self.name)(*args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/stage.py", line 528, in expand_archive
self.fetcher.expand()
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 73, in wrapper
return fun(self, *args, **kwargs)
File "/home/ilumsden/spack/lib/spack/spack/fetch_strategy.py", line 434, in expand
decompress(self.archive_file)
File "/home/ilumsden/spack/lib/spack/spack/util/executable.py", line 189, in __call__
proc.returncode, long_msg)
spack.util.executable.ProcessError: Command exited with status 2:
'/usr/bin/tar' '-xf' '/tmp/ilumsden/spack-stage/spack-stage-cloverleaf-1.1-bq756364x6lklgv2g75cylfr6tbomrxg/CloverLeaf-1.1.tar.gz'
Platform and user environment
On my personal laptop:
$ uname -a
Linux ilumsden-XPS-15-9560 5.0.0-37-generic #40~18.04.1-Ubuntu SMP Thu Nov 14 12:06:39 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -d
Description: Ubuntu 18.04.3 LTS
On UTK's Power9 Cluster:
$ uname -a
Linux tellico-master0 4.14.0-115.10.1.el7a.ppc64le #1 SMP Wed Jun 26 09:32:17 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux
$ lsb_release -d
Description: Red Hat Enterprise Linux Server release 7.6 (Maipo)
Additional information
General information
This issue is essentially an expanded follow-up to #14340.
I've found the following packages from Mantevo that produce checksum failures and cannot be extracted from their tarballs during install:
cloverleafminimdminighostminismac2dminixycehpccgpathfindertealeafThere are several things that these packages have in common:
curlseems to be the Git LFS pointer file. However, I'm still not positive that this is the case.oidfield matches the SHA256 checksum that is in the Spack package file.sha256sumon this pointer file.Additionally, when I tried manually extracting the actual tarball for
cloverleafusingcurlon the command line (following these instructions), I got the following:To keep this issue from getting too lengthy, I'm only going to include the output from trying to install
cloverleafbelow, but I have gotten the same errors for all of the above packages.Spack version
Steps to reproduce the issue
Errors
On my personal laptop:
On UTK's Power9 Cluster:
Platform and user environment
On my personal laptop:
On UTK's Power9 Cluster:
Additional information
General information
spack --versionand reported the version of Spack