chore: reenable py313 by zewenli98 · Pull Request #3455 · pytorch/TensorRT

zewenli98 · 2025-03-31T20:20:13Z

Description

Reenabled Python 3.13 builds since TensorRT has 3.13 wheels available now for TensorRT 10.9.

Type of change

New feature (non-breaking change which adds functionality)

Checklist:

My code follows the style guidelines of this project (You can use the linters)
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas and hacks
I have made corresponding changes to the documentation
I have added tests to verify my fix or my feature
New and existing unit tests pass locally with my changes
I have added the relevant labels to my PR in so that relevant reviewers are notified

.github/scripts/filter-matrix.py

pytorch-bot · 2025-03-31T21:56:32Z

No ciflow labels are configured for this repo.
For information on how to enable CIFlow bot see this wiki

HolyWu · 2025-04-01T06:13:12Z

.github/workflows/build-test-linux.yml

+      - name: Install Rust
+        run: |
+          curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+          source $HOME/.cargo/env


I think you don't need to install rust. Instead, you should try to bump the version of transformers in tests/py/requirements.txt. transformers 4.40.2 depended on tokenizers 0.19.x, but tokenizers 0.19.x didn't have wheels for cp313. That's why pip downloaded the source archive of tokenizers and then compiled unsuccessfully.

The updated transformers doesn't match the results of BERT, compared with old transformers and TRT, so we fixed the version here. Any idea?

The test_bert_base_uncased in test_models and test_models_export was flawed. The inputs were defined as torch.randint(0, 1, ...), which returned a meaningless tensor filled with all zeros because the value of the second arg high is exclusive. To get a useful tensor the high arg should be at least 2, like https://pytorch.org/TensorRT/tutorials/_rendered_examples/dynamo/torch_compile_transformers_example.html.

.github/scripts/generate_binary_build_matrix.py

tests/py/requirements.txt

peri044

LGTM

.github/scripts/generate_binary_build_matrix.py

peri044 · 2025-04-17T20:37:31Z

tests/py/requirements.txt

-transformers==4.40.2
-nvidia-modelopt[deploy,hf,torch]~=0.17.0
+transformers==4.49.0
+nvidia-modelopt[deploy,hf,torch]~=0.17.0; python_version < "3.13"


modelopt doesn't support 3.13 yet ? So, quantization also won't work right in Python 3.13?

right, I think we've already directly skip quantization tests like

TensorRT/tests/py/dynamo/models/test_models_export.py

Lines 201 to 220 in d3eb817

@unittest.skipIf(

not importlib.util.find_spec("modelopt"),

"ModelOpt is required to run this test",

)

@pytest.mark.unit

def test_base_fp8(ir):

import modelopt.torch.quantization as mtq

from modelopt.torch.quantization.utils import export_torch_mode

class SimpleNetwork(torch.nn.Module):

def __init__(self):

super(SimpleNetwork, self).__init__()

self.linear1 = torch.nn.Linear(in_features=10, out_features=5)

self.linear2 = torch.nn.Linear(in_features=5, out_features=1)

def forward(self, x):

x = self.linear1(x)

x = torch.nn.ReLU()(x)

x = self.linear2(x)

return x

Users may not be aware of it though

narendasan · 2025-04-17T20:54:32Z

py/torch_tensorrt/_features.py

    return wrapper


+def needs_refit(f: Callable[..., Any]) -> Callable[..., Any]:


Maybe we can make this a bit more generic (like for any feature in the FeatureSet)

narendasan

Think it mostly looks good, I think we should think about making the requires decorator a bit more generic and unify needs_torch_tensorrt_runtime and needs_refit, maybe something like needs_feature. Maybe something for after 2.7

zewenli98 · 2025-04-17T21:06:11Z

Think it mostly looks good, I think we should think about making the requires decorator a bit more generic and unify needs_torch_tensorrt_runtime and needs_refit, maybe something like needs_feature. Maybe something for after 2.7

sure, will think about it for the next release.

zewenli98 requested a review from lanluo-nvidia March 31, 2025 20:20

zewenli98 self-assigned this Mar 31, 2025

facebook-github-bot added the cla signed label Mar 31, 2025

zewenli98 marked this pull request as draft March 31, 2025 20:20

zewenli98 requested a review from narendasan March 31, 2025 21:24

narendasan reviewed Mar 31, 2025

View reviewed changes

.github/scripts/filter-matrix.py Outdated Show resolved Hide resolved

narendasan added the ciflow/binaries/all Build for all Python Versions label Mar 31, 2025

narendasan linked an issue Mar 31, 2025 that may be closed by this pull request

✨[Feature] py3.13 wheels #3454

Closed

zewenli98 force-pushed the reenable_py313 branch from ff1c583 to ba764bf Compare March 31, 2025 22:43

github-actions bot added the component: tests Issues re: Tests label Apr 1, 2025

HolyWu reviewed Apr 1, 2025

View reviewed changes

.github/scripts/generate_binary_build_matrix.py Show resolved Hide resolved

zewenli98 force-pushed the reenable_py313 branch 2 times, most recently from 323e177 to 404365a Compare April 1, 2025 19:54

narendasan reviewed Apr 2, 2025

View reviewed changes

tests/py/requirements.txt Outdated Show resolved Hide resolved

github-actions bot added component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 3, 2025

zewenli98 added 9 commits April 2, 2025 17:13

reenable py313

2caf324

clear py filter

0ccb6eb

fix python version req

177ed03

upgrade transformers

ab88c60

keep transformers==4.40.2

8b04754

fix

fe9faa3

try transformers==4.49.0

41546a8

add python 3.13t in workflow

9358250

fix

f60e091

zewenli98 force-pushed the reenable_py313 branch from 8b6f880 to f60e091 Compare April 3, 2025 00:13

github-actions bot removed the component: conversion Issues re: Conversion stage label Apr 3, 2025

github-actions bot removed component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 3, 2025

narendasan added cherry-pick To cherry-pick to a release branch needs-release-cherrypick labels Apr 4, 2025

github-actions bot requested a review from narendasan April 4, 2025 00:35

narendasan removed the cherry-pick To cherry-pick to a release branch label Apr 4, 2025

update bdist_wheel

6e597ad

github-actions bot added the component: build system Issues re: Build system label Apr 9, 2025

zewenli98 added 7 commits April 8, 2025 21:46

small fix

9455634

attempt fix

a0b0e93

attempt fix

822e2fe

attempt fix

3ad2156

revert to not support py313t

7e864ad

fix

826b7b2

fix not on the same device error

da0e3a7

github-actions bot added component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Apr 11, 2025

zewenli98 added 2 commits April 16, 2025 12:41

disable refit and engine caching for py313

9089c1c

disable related tests

e06a930

zewenli98 marked this pull request as ready for review April 17, 2025 20:24

peri044 approved these changes Apr 17, 2025

View reviewed changes

narendasan reviewed Apr 17, 2025

View reviewed changes

zewenli98 merged commit 72947c5 into main Apr 17, 2025
461 of 462 checks passed

zewenli98 mentioned this pull request Apr 18, 2025

Cherrypick #3455 for release/2.7 #3480

Merged

7 tasks

zewenli98 added a commit that referenced this pull request Apr 18, 2025

Cherrypick #3455 for release/2.7 (#3480)

380bb57

zewenli98 deleted the reenable_py313 branch January 12, 2026 21:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: reenable py313#3455

chore: reenable py313#3455
zewenli98 merged 22 commits intomainfrom
reenable_py313

zewenli98 commented Mar 31, 2025

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 31, 2025

Uh oh!

HolyWu Apr 1, 2025

Uh oh!

zewenli98 Apr 2, 2025

Uh oh!

HolyWu Apr 2, 2025

Uh oh!

Uh oh!

Uh oh!

peri044 left a comment

Uh oh!

Uh oh!

peri044 Apr 17, 2025

Uh oh!

zewenli98 Apr 17, 2025

Uh oh!

narendasan Apr 17, 2025

Uh oh!

narendasan left a comment

Uh oh!

zewenli98 commented Apr 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

	@unittest.skipIf(
	not importlib.util.find_spec("modelopt"),
	"ModelOpt is required to run this test",
	)
	@pytest.mark.unit
	def test_base_fp8(ir):
	import modelopt.torch.quantization as mtq
	from modelopt.torch.quantization.utils import export_torch_mode

	class SimpleNetwork(torch.nn.Module):
	def __init__(self):
	super(SimpleNetwork, self).__init__()
	self.linear1 = torch.nn.Linear(in_features=10, out_features=5)
	self.linear2 = torch.nn.Linear(in_features=5, out_features=1)

	def forward(self, x):
	x = self.linear1(x)
	x = torch.nn.ReLU()(x)
	x = self.linear2(x)
	return x

		return wrapper


		def needs_refit(f: Callable[..., Any]) -> Callable[..., Any]:

Conversation

zewenli98 commented Mar 31, 2025

Description

Type of change

Checklist:

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 31, 2025

Uh oh!

HolyWu Apr 1, 2025

Choose a reason for hiding this comment

Uh oh!

zewenli98 Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

HolyWu Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

peri044 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

peri044 Apr 17, 2025

Choose a reason for hiding this comment

Uh oh!

zewenli98 Apr 17, 2025

Choose a reason for hiding this comment

Uh oh!

narendasan Apr 17, 2025

Choose a reason for hiding this comment

Uh oh!

narendasan left a comment

Choose a reason for hiding this comment

Uh oh!

zewenli98 commented Apr 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants