🐛 Bug

fairscale uses pyproject.toml and opts in to the new setuptools.build_meta build mechanism with build isolation

Build isolation means that pip will create an isolated build environment and only provide the packages listed in

These packages are installed from pypi. Unfortunately that also implies that fairscale is built against a different version of torch than what is available at runtime. This may cause crashes if the version of torch available at build time is ABI incompatible to the version at runtime. Besides the ABI incompatibility issue, this also means that pip install fairscale will usually trigger a 700MB+ download of the default torch package from pypi, even though the user may already have installed torch from https://download.pytorch.org/whl/torch_stable.html or by compiling from source.

Thus I'd suggest to opt out of build isolation feature by adopting build-backend = "setuptools.build_meta:__legacy__"
It is a a “compatibility mode” PEP 517 backend added in pypa/setuptools#1652.

I previously opened an issue with pip about this behavior. You can read more at pypa/pip#9439.

To Reproduce

pip install --verbose fairscale

See the log

[...]
  Collecting torch>=1.4.0
    Created temporary directory: /tmp/pip-unpack-9i9miqsz
    Getting credentials from keyring for files.pythonhosted.org
    Looking up "https://files.pythonhosted.org/packages/1d/a9/f349273a0327fdf20a73188c9c3aa7dbce68f86fad422eadd366fd2ed7a0/torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl" in the cache
    No cache entry available
    Starting new HTTPS connection (1): files.pythonhosted.org:443
    https://files.pythonhosted.org:443 "GET /packages/1d/a9/f349273a0327fdf20a73188c9c3aa7dbce68f86fad422eadd366fd2ed7a0/torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl HTTP/1.1" 200 776818711
    Downloading torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl (776.8 MB)
^C  Installing build dependencies ... canceled
ERROR: Operation cancelled by user