Skip to content

[OpenReg] Implement device autoload mechanism#158555

Closed
can-gaa-hou wants to merge 10 commits intopytorch:mainfrom
can-gaa-hou:openreg
Closed

[OpenReg] Implement device autoload mechanism#158555
can-gaa-hou wants to merge 10 commits intopytorch:mainfrom
can-gaa-hou:openreg

Conversation

@can-gaa-hou
Copy link
Contributor

@can-gaa-hou can-gaa-hou commented Jul 17, 2025

Implement OpenReg device autoload mechanism

Overview

The Autoload mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., torch.backends) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

Motivation

This PR aims to apply device autoload mechanism to the OpenReg module with some simple changes.

Change

Before

import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)

After

import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)

cc @fffrog @albanD

@pytorch-bot
Copy link

pytorch-bot bot commented Jul 17, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158555

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit ca3e0bf with merge base b16d3f4 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the topic: not user facing topic category label Jul 17, 2025
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jul 17, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

@@ -125,7 +125,7 @@ graph LR

### Autoload
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a section for Autoload:

Take a introduction about background and design of Autoload, and add detailed description to show how the new accelerator to use Autoload mechinasm

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I have added more details about Autoload mechanism. Please let me know if there any other problems :)

@fffrog
Copy link
Collaborator

fffrog commented Aug 5, 2025

What iss more , please refer to this PR to find out the tests about Autoload and migrate all of those to torch_openreg please.

@can-gaa-hou can-gaa-hou force-pushed the openreg branch 2 times, most recently from c6b901b to 223dfd9 Compare August 8, 2025 02:08
@can-gaa-hou can-gaa-hou requested a review from fffrog August 8, 2025 02:22
@can-gaa-hou can-gaa-hou marked this pull request as ready for review August 11, 2025 02:28
@can-gaa-hou can-gaa-hou requested a review from a team as a code owner August 11, 2025 02:28
@jerryzh168 jerryzh168 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Aug 13, 2025
@fffrog
Copy link
Collaborator

fffrog commented Aug 13, 2025

@can-gaa-hou Sorry for the late reply.

To get this PR reviewed better, we may need to split it into several parts:

  • Add implementation, readme and tutorial of Autoload functionality
  • Migrate autoloading-related tests to torch_openreg

You should base your commit on this PR, as it builds the basic framework for the tutorial, which this PR will uses.

can-gaa-hou and others added 7 commits September 4, 2025 20:29
Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
…setup.py

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
…README.md

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
…README.md

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
@can-gaa-hou can-gaa-hou requested a review from fffrog September 5, 2025 09:03
@can-gaa-hou can-gaa-hou changed the title Implement OpenReg device autoload mechanism [OpenReg] Implement device autoload mechanism Sep 8, 2025
Copy link
Collaborator

@fffrog fffrog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much.

@albanD, could you please help to take a look at this one?

Copy link
Collaborator

@albanD albanD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great.
Thanks for the special care to the documentation!

@can-gaa-hou
Copy link
Contributor Author

@pytorchbot merge

@pytorch-bot pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 12, 2025
@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

This PR (#158555) was merged in 9519152 but it is still open, likely due to a Github bug, so mergebot is closing it manually. If you think this is a mistake, please feel free to reopen and contact Dev Infra.

SherlockNoMad pushed a commit to SherlockNoMad/pytorch that referenced this pull request Sep 12, 2025
# Implement OpenReg device autoload mechanism

## Overview
The **Autoload** mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., `torch.backends`) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

## Motivation

This PR aims to apply [device autoload mechanism](pytorch#122468) to the OpenReg module with some simple changes.

## Change
### Before
```python
import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```
### After
```python
import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```

Pull Request resolved: pytorch#158555
Approved by: https://github.com/FFFrog, https://github.com/albanD

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
markc-614 pushed a commit to markc-614/pytorch that referenced this pull request Sep 17, 2025
# Implement OpenReg device autoload mechanism

## Overview
The **Autoload** mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., `torch.backends`) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

## Motivation

This PR aims to apply [device autoload mechanism](pytorch#122468) to the OpenReg module with some simple changes.

## Change
### Before
```python
import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```
### After
```python
import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```

Pull Request resolved: pytorch#158555
Approved by: https://github.com/FFFrog, https://github.com/albanD

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025
# Implement OpenReg device autoload mechanism

## Overview
The **Autoload** mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., `torch.backends`) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

## Motivation

This PR aims to apply [device autoload mechanism](pytorch#122468) to the OpenReg module with some simple changes.

## Change
### Before
```python
import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```
### After
```python
import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```

Pull Request resolved: pytorch#158555
Approved by: https://github.com/FFFrog, https://github.com/albanD

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025
# Implement OpenReg device autoload mechanism

## Overview
The **Autoload** mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., `torch.backends`) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

## Motivation

This PR aims to apply [device autoload mechanism](pytorch#122468) to the OpenReg module with some simple changes.

## Change
### Before
```python
import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```
### After
```python
import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```

Pull Request resolved: pytorch#158555
Approved by: https://github.com/FFFrog, https://github.com/albanD

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025
# Implement OpenReg device autoload mechanism

## Overview
The **Autoload** mechanism in PyTorch simplifies the integration of third-party device backends by enabling automatic discovery and initialization at runtime. Traditionally, integrating a new backend required explicit imports or manual initialization, which could be cumbersome and error-prone. With Autoload, PyTorch dynamically detects and initializes device backends, providing a seamless user experience.

This mechanism leverages Python entry points (e.g., `torch.backends`) and dynamic module loading. When PyTorch starts, it scans for registered entry points and invokes their initialization hooks, ensuring that all available backends are ready for use without requiring explicit imports.

## Motivation

This PR aims to apply [device autoload mechanism](pytorch#122468) to the OpenReg module with some simple changes.

## Change
### Before
```python
import torch
import torch_openreg

x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```
### After
```python
import torch

# No need to import torch_openreg manually!
x = torch.tensor([1, 2, 3], device="openreg")
print(x)
```

Pull Request resolved: pytorch#158555
Approved by: https://github.com/FFFrog, https://github.com/albanD

Co-authored-by: Jiawei Li <ljw1101.vip@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk Trigger trunk jobs on your pull request Merged open source topic: not user facing topic category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants