module 'numcodecs' has no attribute 'Shuffle' #195
Description
I'm a newbie here, so apologies if I'm missing something obvious! I'm trying to read a NetCDF4 file over http(s) and am getting the error message "module 'numcodecs' has no attribute 'Shuffle'". This has happened with a range of files.
I've done quite a lot of googling about this error but haven't found a solution (or at least, not one that I understood!) Perhaps this is the closest discussion: zarr-developers/numcodecs#260.
I'm not sure if this is an issue in kerchunk.hdf or whether it's deeper in numcodecs. In case it helps, I'm on an M1 Mac - sometimes this causes issues because there's a missing binary (ARM) dependency somewhere in the tree.
Here's some code that reproduces the error on my system. I'm using one of the free NetCDF4 files provided by Unidata as the example:
import json
import fsspec
import kerchunk.hdf
URL = "https://www.unidata.ucar.edu/software/netcdf/examples/test_echam_spectral-deflated.nc"
with fsspec.open(URL, "rb") as f:
h5chunks = kerchunk.hdf.SingleHdf5ToZarr(f, URL, inline_threshold=100)
print(json.dumps(h5chunks.translate()))Traceback:
Traceback (most recent call last):
File "/Users/me/Code/kerchunk-test/test2.py", line 12, in <module>
print(json.dumps(h5chunks.translate()))
File "/Users/me/opt/miniconda3/envs/kerchunk-test/lib/python3.10/site-packages/kerchunk/hdf.py", line 73, in translate
self._h5f.visititems(self._translator)
File "/Users/me/opt/miniconda3/envs/kerchunk-test/lib/python3.10/site-packages/h5py/_hl/group.py", line 613, in visititems
return h5o.visit(self.id, proxy)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5o.pyx", line 355, in h5py.h5o.visit
File "h5py/h5o.pyx", line 302, in h5py.h5o.cb_obj_simple
File "/Users/me/opt/miniconda3/envs/kerchunk-test/lib/python3.10/site-packages/h5py/_hl/group.py", line 612, in proxy
return func(name, self[name])
File "/Users/me/opt/miniconda3/envs/kerchunk-test/lib/python3.10/site-packages/kerchunk/hdf.py", line 182, in _translator
filters.append(numcodecs.Shuffle(elementsize=h5obj.dtype.itemsize))
AttributeError: module 'numcodecs' has no attribute 'Shuffle'