[Windows] PyTorch import errors from torch.distributed (e.g. running ComfyUI)

> [!IMPORTANT]
> SUMMARY:
> 
> If you see
>
> ```
> ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
> ````
>
> This is due to `import torch.distributed.tensor` failing but being called by the `transformers` package even if `if torch.distributed.is_available()` returns false. A fix has been merged but is not yet included in a stable release of that package: https://github.com/huggingface/transformers/commit/a5fac1c394eafb07312519fcef48ffe8f1a247dc.
>
> The workarounds are _either_
> * For Python < 3.13, install the last working version of `transformers`:
>     ```
>     pip install transformers==4.41.2
>     ```
> * For any Python version, install the latest source which includes a fix:
>     ```
>     pip install git+https://github.com/huggingface/transformers
>     ```

---

Forking from https://github.com/ROCm/TheRock/issues/910#issuecomment-3156199454.

Setup:

```bash
(3.13.venv) λ python -m pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx110X-dgpu/ torch torchvision torchaudio
...
(3.13.venv) λ pip freeze
filelock==3.18.0
fsspec==2025.5.1
Jinja2==3.1.6
MarkupSafe==3.0.2
mpmath==1.3.0
networkx==3.5
numpy==2.3.1
pillow==11.3.0
rocm==7.0.0rc20250805
rocm-sdk-core==7.0.0rc20250805
rocm-sdk-libraries-gfx110X-dgpu==7.0.0rc20250805
setuptools==80.9.0
sympy==1.14.0
torch==2.9.0a0+rocm7.0.0rc20250805
torchaudio==2.8.0a0+rocm7.0.0rc20250805
torchvision==0.24.0a0+rocm7.0.0rc20250805
typing_extensions==4.14.1
```

Then trying to `import torch.distributed.tensor` fails:
```bash
(3.13.venv) λ python
Python 3.13.5 (tags/v3.13.5:6cb20a2, Jun 11 2025, 16:15:46) [MSC v.1943 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch.distributed.tensor
Traceback (most recent call last):
  File "<python-input-0>", line 1, in <module>
    import torch.distributed.tensor
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\tensor\__init__.py", line 4, in <module>
    import torch.distributed.tensor._ops  # force import all built-in dtensor ops
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\tensor\_ops\__init__.py", line 2, in <module>
    from ._conv_ops import *  # noqa: F403
    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\tensor\_ops\_conv_ops.py", line 5, in <module>
    from torch.distributed.tensor._dtensor_spec import DTensorSpec, TensorMeta
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\tensor\_dtensor_spec.py", line 6, in <module>
    from torch.distributed.tensor.placement_types import (
    ...<4 lines>...
    )
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\tensor\placement_types.py", line 8, in <module>
    import torch.distributed._functional_collectives as funcol
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\_functional_collectives.py", line 9, in <module>
    import torch.distributed.distributed_c10d as c10d
  File "D:\scratch\therock\3.13.venv\Lib\site-packages\torch\distributed\distributed_c10d.py", line 23, in <module>
    from torch._C._distributed_c10d import (
    ...<22 lines>...
    )
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
```

Users report that https://github.com/comfyanonymous/ComfyUI triggers this code path on launch.

---

Copying one of my comments from https://github.com/ROCm/TheRock/issues/1179:

> I would not expect distributed APIs to work until we somehow enable GLOO (the collective communications library used by PyTorch: https://github.com/pytorch/gloo) on Windows:
> 
> https://github.com/ROCm/TheRock/blob/7615bf9fe01815bf0bc3591f9eb859c169f5e9e8/external-builds/pytorch/build_prod_wheels.py#L371-L373
> 
> For that, we would need RCCL (the ROCm Communication Collectives Library), which is currently unsupported on Windows: https://github.com/ROCm/TheRock/blob/main/docs/development/windows_support.md#supported-subprojects
> 
> Maybe we can improve the error message there and note this as a known feature gap until then.

Taking that a step further, maybe we can help this degrade gracefully instead of failing?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Windows] PyTorch import errors from torch.distributed (e.g. running ComfyUI) #1202

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# GLOO enabled for only Linux
	if not is_windows:
	env["USE_GLOO"] = "ON"

[Windows] PyTorch import errors from torch.distributed (e.g. running ComfyUI) #1202

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions