Scipy's gmres fails to converge with Dask Array #8702
Description
What happened:
During a review of #8694, I noticed that scipy's gmres fails to converge on a solution with even a small dask array after several minutes. The task graph grows seemingly boundlessly
What you expected to happen:
gmres should converge seemingly instantly
Minimal Complete Verifiable Example:
import numpy as np
from scipy.sparse.linalg import gmres
from scipy.sparse.linalg._interface import MatrixLinearOperator
from dask.distributed import Client
import dask.array as da
np.random.seed(0)
x_np = np.random.random(size=(40, 40))
with Client(n_workers=4) as client:
x = da.from_array(x_np, chunks=(4, 4))
A = MatrixLinearOperator(x)
b = np.random.random(40)
result = gmres(A, b)Anything else we need to know?:
Environment:
- Dask version: 2022.1.1
- Python version: 3.9.10
- Scipy version: 1.18.0
- Operating System: Mac OS 12.2 (21D49) with the M1 Pro chip
- Install method (conda, pip, source): conda (mambaforge in particular)
Full env
# packages in environment at /Users/paul/mambaforge/envs/hydro:
#
# Name Version Build Channel
aiohttp 3.8.1 py39h5161555_0 conda-forge
aiosignal 1.2.0 pyhd8ed1ab_0 conda-forge
anyio 3.5.0 py39h2804cbe_0 conda-forge
appnope 0.1.2 py39h2804cbe_2 conda-forge
argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge
argon2-cffi-bindings 21.2.0 py39h5161555_1 conda-forge
asttokens 2.0.5 pyhd8ed1ab_0 conda-forge
async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge
attrs 21.4.0 pyhd8ed1ab_0 conda-forge
babel 2.9.1 pyh44b312d_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
black 22.1.0 pyhd8ed1ab_0 conda-forge
bleach 4.1.0 pyhd8ed1ab_0 conda-forge
bokeh 2.4.2 py39h2804cbe_0 conda-forge
brotli 1.0.9 h3422bc3_6 conda-forge
brotli-bin 1.0.9 h3422bc3_6 conda-forge
brotlipy 0.7.0 py39h5161555_1003 conda-forge
bzip2 1.0.8 h3422bc3_4 conda-forge
ca-certificates 2021.10.8 h4653dfc_0 conda-forge
certifi 2021.10.8 py39h2804cbe_1 conda-forge
cffi 1.15.0 py39h52b1de0_0 conda-forge
charset-normalizer 2.0.11 pyhd8ed1ab_0 conda-forge
click 8.0.3 py39h2804cbe_1 conda-forge
cloudpickle 2.0.0 pyhd8ed1ab_0 conda-forge
cloudside 0.1 dev_0 <develop>
cryptography 36.0.1 py39hfb8cd70_0 conda-forge
cycler 0.11.0 pyhd8ed1ab_0 conda-forge
cytoolz 0.11.2 py39h5161555_1 conda-forge
dask 2022.1.1 pyhd8ed1ab_0 conda-forge
dask-core 2022.1.1 pyhd8ed1ab_0 conda-forge
dask-labextension 5.2.0 pyhd8ed1ab_0 conda-forge
dataclasses 0.8 pyhc8e2a94_3 conda-forge
debugpy 1.5.1 py39hfb83b0d_0 conda-forge
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
distributed 2022.1.1 py39h2804cbe_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
executing 0.8.2 pyhd8ed1ab_0 conda-forge
flit-core 3.6.0 pyhd8ed1ab_0 conda-forge
fonttools 4.29.1 py39h5161555_0 conda-forge
freetype 2.10.4 h17b34a0_1 conda-forge
fribidi 1.0.10 h27ca646_0 conda-forge
frozenlist 1.3.0 py39h5161555_0 conda-forge
fsspec 2022.1.0 pyhd8ed1ab_0 conda-forge
giflib 5.2.1 h27ca646_2 conda-forge
heapdict 1.0.1 py_0 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
importlib-metadata 4.10.1 py39h2804cbe_0 conda-forge
importlib_resources 5.4.0 pyhd8ed1ab_0 conda-forge
iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge
ipykernel 6.9.0 py39h32adebf_0 conda-forge
ipython 8.0.1 py39h2804cbe_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
jbig 2.1 h3422bc3_2003 conda-forge
jedi 0.18.1 py39h2804cbe_0 conda-forge
jinja2 3.0.3 pyhd8ed1ab_0 conda-forge
jpeg 9e h3422bc3_0 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 4.4.0 pyhd8ed1ab_0 conda-forge
jupyter-server-proxy 3.2.1 pyhd8ed1ab_0 conda-forge
jupyter_client 7.1.2 pyhd8ed1ab_0 conda-forge
jupyter_core 4.9.1 py39h2804cbe_1 conda-forge
jupyter_server 1.13.5 pyhd8ed1ab_0 conda-forge
jupyterlab 3.2.9 pyhd8ed1ab_0 conda-forge
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
jupyterlab_server 2.10.3 pyhd8ed1ab_0 conda-forge
kiwisolver 1.3.2 py39h4d2d688_1 conda-forge
lcms2 2.12 had6a04f_0 conda-forge
lerc 3.0 hbdafb3b_0 conda-forge
libblas 3.9.0 13_osxarm64_openblas conda-forge
libbrotlicommon 1.0.9 h3422bc3_6 conda-forge
libbrotlidec 1.0.9 h3422bc3_6 conda-forge
libbrotlienc 1.0.9 h3422bc3_6 conda-forge
libcblas 3.9.0 13_osxarm64_openblas conda-forge
libcxx 12.0.1 h168391b_1 conda-forge
libdeflate 1.8 h3422bc3_0 conda-forge
libffi 3.4.2 h3422bc3_5 conda-forge
libgfortran 5.0.0.dev0 11_0_1_hf114ba7_23 conda-forge
libgfortran5 11.0.1.dev0 hf114ba7_23 conda-forge
liblapack 3.9.0 13_osxarm64_openblas conda-forge
libopenblas 0.3.18 openmp_h5dd58f0_0 conda-forge
libpng 1.6.37 hf7e6567_2 conda-forge
libsodium 1.0.18 h27ca646_1 conda-forge
libtiff 4.3.0 h74060c4_2 conda-forge
libwebp 1.2.2 h0d20362_0 conda-forge
libwebp-base 1.2.2 h3422bc3_1 conda-forge
libxcb 1.13 h9b22ae9_1004 conda-forge
libzlib 1.2.11 hee7b306_1013 conda-forge
llvm-openmp 12.0.1 hf3c4609_1 conda-forge
locket 0.2.0 py_2 conda-forge
lz4-c 1.9.3 hbdafb3b_1 conda-forge
markupsafe 2.0.1 py39h5161555_1 conda-forge
matplotlib 3.5.1 py39hdf13c20_0 conda-forge
matplotlib-base 3.5.1 py39h5aa4fe7_0 conda-forge
matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge
metar 1.9.0 pyhd8ed1ab_0 conda-forge
mistune 0.8.4 py39h5161555_1005 conda-forge
msgpack-python 1.0.3 py39h4d2d688_0 conda-forge
multidict 6.0.2 py39h5161555_0 conda-forge
munkres 1.1.4 pyh9f0ad1d_0 conda-forge
mypy_extensions 0.4.3 py39h2804cbe_4 conda-forge
nbclassic 0.3.5 pyhd8ed1ab_0 conda-forge
nbclient 0.5.10 pyhd8ed1ab_1 conda-forge
nbconvert 6.4.1 py39h2804cbe_0 conda-forge
nbformat 5.1.3 pyhd8ed1ab_0 conda-forge
ncurses 6.3 hc470f4d_0 conda-forge
nest-asyncio 1.5.4 pyhd8ed1ab_0 conda-forge
notebook 6.4.8 pyha770c72_0 conda-forge
numpy 1.22.2 py39h61a45d2_0 conda-forge
openjpeg 2.4.0 h062765e_1 conda-forge
openssl 1.1.1l h3422bc3_0 conda-forge
packaging 21.3 pyhd8ed1ab_0 conda-forge
pandas 1.4.0 py39h7f752ed_0 conda-forge
pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.2.0 pyhd8ed1ab_0 conda-forge
pathspec 0.9.0 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.0.1 py39hcb29f89_0 conda-forge
pip 22.0.3 pyhd8ed1ab_0 conda-forge
platformdirs 2.4.1 pyhd8ed1ab_1 conda-forge
pluggy 1.0.0 py39h2804cbe_2 conda-forge
prometheus_client 0.13.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.26 pyha770c72_0 conda-forge
psutil 5.9.0 py39h5161555_0 conda-forge
pthread-stubs 0.4 h27ca646_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
py 1.11.0 pyh6c4a22f_0 conda-forge
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pygments 2.11.2 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.7 pyhd8ed1ab_0 conda-forge
pyrsistent 0.18.1 py39h5161555_0 conda-forge
pysocks 1.7.1 py39h2804cbe_4 conda-forge
pytest 7.0.0 py39h2804cbe_0 conda-forge
python 3.9.10 hd16f9c5_2_cpython conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.9 2_cp39 conda-forge
pytz 2021.3 pyhd8ed1ab_0 conda-forge
pyyaml 6.0 py39h5161555_3 conda-forge
pyzmq 22.3.0 py39h02c6a76_1 conda-forge
readline 8.1 hedafd6a_0 conda-forge
requests 2.27.1 pyhd8ed1ab_0 conda-forge
scipy 1.8.0 py39h5060c3b_1 conda-forge
send2trash 1.8.0 pyhd8ed1ab_0 conda-forge
setuptools 59.8.0 py39h2804cbe_0 conda-forge
simpervisor 0.4 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sniffio 1.2.0 py39h2804cbe_2 conda-forge
sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge
sqlite 3.37.0 h72a2b83_0 conda-forge
stack_data 0.1.4 pyhd8ed1ab_0 conda-forge
tblib 1.7.0 pyhd8ed1ab_0 conda-forge
terminado 0.13.1 py39h2804cbe_0 conda-forge
testpath 0.5.0 pyhd8ed1ab_0 conda-forge
tk 8.6.11 he1e0b03_1 conda-forge
tomli 2.0.0 pyhd8ed1ab_1 conda-forge
toolz 0.11.2 pyhd8ed1ab_0 conda-forge
tornado 6.1 py39h5161555_2 conda-forge
traitlets 5.1.1 pyhd8ed1ab_0 conda-forge
typed-ast 1.5.2 py39h5161555_0 conda-forge
typing-extensions 4.0.1 hd8ed1ab_0 conda-forge
typing_extensions 4.0.1 pyha770c72_0 conda-forge
tzdata 2021e he74cb21_0 conda-forge
unicodedata2 14.0.0 py39h5161555_0 conda-forge
urllib3 1.26.8 pyhd8ed1ab_1 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
websocket-client 1.2.3 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd8ed1ab_0 conda-forge
xorg-libxau 1.0.9 h27ca646_0 conda-forge
xorg-libxdmcp 1.1.3 h27ca646_0 conda-forge
xz 5.2.5 h642e427_1 conda-forge
yaml 0.2.5 h3422bc3_2 conda-forge
yarl 1.7.2 py39h5161555_1 conda-forge
zeromq 4.3.4 hbdafb3b_1 conda-forge
zict 2.0.0 py_0 conda-forge
zipp 3.7.0 pyhd8ed1ab_1 conda-forge
zlib 1.2.11 hee7b306_1013 conda-forge
zstd 1.5.2 h861e0a7_0 conda-forge