Loading data with pydap engine fails due to outdated/missing dependency #6648
Description
What happened?
Hi,
I was trying to load a dataset with the pydap engine, but unfortunately it failed due to a mis-match of arguments in xarray's pydap backend and the used version of pydap (3.2.2).
What did you expect to happen?
Retrieving a nice dataset 😄
Minimal Complete Verifiable Example
import xarray as xr
xr.open_dataset('http://test.opendap.org/dap/data/nc/coads_climatology.nc', engine='pydap')MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
TypeError Traceback (most recent call last)
<ipython-input-4-36e60ce43c69> in <module>
----> 1 xr.open_dataset('http://test.opendap.org/dap/data/nc/coads_climatology.nc', engine='pydap')
~/mambaforge/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, backend_kwargs, *args, **kwargs)
493
494 overwrite_encoded_chunks = kwargs.pop("overwrite_encoded_chunks", None)
--> 495 backend_ds = backend.open_dataset(
496 filename_or_obj,
497 drop_variables=drop_variables,
~/mambaforge/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/pydap_.py in open_dataset(self, filename_or_obj, mask_and_scale, decode_times, concat_characters, decode_coords, drop_variables, use_cftime, decode_timedelta, application, session, output_grid, timeout, verify, user_charset)
163 ):
164
--> 165 store = PydapDataStore.open(
166 url=filename_or_obj,
167 application=application,
~/mambaforge/envs/how_to_eurec4a/lib/python3.8/site-packages/xarray/backends/pydap_.py in open(cls, url, application, session, output_grid, timeout, verify, user_charset)
112 user_charset = "ascii"
113
--> 114 ds = pydap.client.open_url(
115 url=url,
116 application=application,
TypeError: open_url() got an unexpected keyword argument 'verify'Anything else we need to know?
The root-cause of this issue seems to be a missing dependency.
With dfaedb2 the verify-argument has been added to xarray/backends/pydap_.py, which is supported by pydap since late 2018 (pydap/pydap#112). However, only recently a new release of pydap has been published and incorporates these changes now.
Version 3.3.0, released on 1. Feb. 2022 (includes verify)
Version 3.2.2 released on 25. May 2017 (has no verify)
Unfortunately, version 3.3.0 is not yet available on pypi and only on conda-forge.
I couldn't find any pins or limitations on versions in e.g. requirements.txt or setup.cfg for non-core dependencies. Should this dependency be introduced somewhere? At least https://github.com/pydata/xarray/blob/e02b1c3f6d18c7afcdf4f78cf3463652b4cc96c9/ci/requirements/min-all-deps.yml needs to be updated I guess.
Environment
Details
INSTALLED VERSIONS ------------------ commit: None python: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:06:49) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1xarray: 2022.3.0
pandas: 1.4.2
numpy: 1.22.3
scipy: 1.8.0
netCDF4: 1.5.8
pydap: installed
h5netcdf: None
h5py: None
Nio: None
zarr: 2.11.1
cftime: 1.6.0
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2022.04.1
distributed: 2022.4.1
matplotlib: 3.5.1
cartopy: 0.20.2
seaborn: None
numbagg: None
fsspec: 2022.5.0
cupy: None
pint: None
sparse: None
setuptools: 61.1.1
pip: 22.0.4
conda: None
pytest: 7.1.2
IPython: 7.29.0
sphinx: 4.5.0