[IR] Create `predecessors()` and `successors()` on `ir.Node` #2022

justinchuby · 2025-01-18T18:49:47Z

Also updated Usage to a named tuple
Implement consumers() on Value

Copilot

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (5)

onnxscript/ir/_core.py:1314

Ensure that value is not None before calling value.producer().

if value is not None and (producer := value.producer()) is not None:

onnxscript/ir/_core.py:1326

[nitpick] Consider replacing the assert statement with a more informative error message or exception.

assert value is not None, "Bug: Output values are not expected to be None"

onnxscript/ir/_core.py:1059

[nitpick] Consider renaming 'Usage' to 'NodeUsage' for better clarity.

class Usage(NamedTuple):

onnxscript/ir/_core.py:1309

Ensure that the 'predecessors' method is covered by tests.

def predecessors(self) -> Sequence[Node]:

onnxscript/ir/_core.py:1321

Ensure that the 'successors' method is covered by tests.

def successors(self) -> Sequence[Node]:

onnxscript/ir/_core.py

codecov · 2025-01-18T18:56:19Z

❌ 70 Tests Failed:

Tests completed	Failed	Passed	Skipped
11840	70	11770	2454

View the top 3 failed tests by shortest run time

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0414_test_gemm_default_scalar_bias

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_gemm_default_scalar_bias'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_gemm_default_scalar_bias' (e=No module named 'tests.onnx_backend_test_code.test_gemm_default_scalar_bias') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_scalar_bias.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_gemm_default_scalar_bias.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT
E   from onnxscript.onnx_opset import opset13
E   
E   @script()
E   def bck_test_gemm_default_scalar_bias(a: FLOAT[2,3], b: FLOAT[3,4], c: FLOAT) -> (FLOAT[2,4]):
E       y = opset13.Gemm(a, b, c)
E       return y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_0988_test_resize_upsample_sizes_nearest_axes_2_3

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_2_3'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_2_3' (e=No module named 'tests.onnx_backend_test_code.test_resize_upsample_sizes_nearest_axes_2_3') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_resize_upsample_sizes_nearest_axes_2_3.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_resize_upsample_sizes_nearest_axes_2_3.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset19
E   
E   @script()
E   def bck_test_resize_upsample_sizes_nearest_axes_2_3(X: FLOAT[1,1,2,2], sizes: INT64[2]) -> (FLOAT[1,1,7,8]):
E       Y = opset19.Resize(X, None, None, sizes, axes=[2, 3], mode='nearest')
E       return Y

onnxscript.backend.onnx_export_test.TestOnnxBackEnd::test_export2python_produces_correct_onnx_script_model_1008_test_scatternd_multiply

Stack Traces | 0.003s run time

onnxscript\backend\onnx_export_test.py:137: in extract_functions
    mod = importlib.import_module(import_name)
C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\importlib\__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'tests.onnx_backend_test_code.test_scatternd_multiply'

The above exception was the direct cause of the following exception:
.nox\test_onnx_weekly\Lib\site-packages\parameterized\parameterized.py:620: in standalone_func
    return func(*(a + p.args), **p.kwargs, **kw)
onnxscript\backend\onnx_export_test.py:271: in test_export2python_produces_correct_onnx_script_model
    functions = extract_functions(backend_test.name, code, self.test_folder)
onnxscript\backend\onnx_export_test.py:139: in extract_functions
    raise AssertionError(
E   AssertionError: Unable to import 'tests.onnx_backend_test_code.test_scatternd_multiply' (e=No module named 'tests.onnx_backend_test_code.test_scatternd_multiply') (file: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_scatternd_multiply.py', absolute path: 'D:\\a\\onnxscript\\onnxscript\\tests\\onnx_backend_test_code\\test_scatternd_multiply.py', current folder: D:\a\onnxscript\onnxscript
E   ---- CONTENT --
E   import numpy
E   from onnx import TensorProto
E   from onnx.helper import make_tensor
E   from onnxscript import script, external_tensor
E   from onnxscript.values import Opset
E   from onnxscript.onnx_types import FLOAT, INT64
E   from onnxscript.onnx_opset import opset18
E   
E   @script()
E   def bck_test_scatternd_multiply(data: FLOAT[4,4,4], indices: INT64[2,1], updates: FLOAT[2,4,4]) -> (FLOAT[4,4,4]):
E       y = opset18.ScatterND(data, indices, updates, reduction='mul')
E       return y

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

onnxscript/ir/_core.py

Copilot

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

onnxscript/ir/_core.py:1646

The word 'addes' is misspelled. It should be 'adds'.

// This addes a small overhead but is better a user experience than

onnxscript/ir/_core_test.py:817

Add a test for the successors method.

    # TODO(justinchuby): Test all methods

onnxscript/ir/_core.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

onnxscript/ir/_core.py

Copilot

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (2)

onnxscript/ir/_core_test.py:834

Ensure that the order of nodes in the predecessors method is deterministic and consistent.

self.assertEqual(self.node.predecessors(), ())

onnxscript/ir/_core_test.py:843

Ensure that the order of nodes in the successors method is deterministic and consistent.

self.assertEqual(self.node.successors(), (self.node_a, self.node_b))

onnxscript/ir/_core.py

justinchuby · 2025-01-21T17:07:13Z

Will merge now and resolve further comments as follow ups. Thanks!

### Description This PR adds fusions for [Google's SigLIP model](https://huggingface.co/google/siglip-base-patch16-224/) and Microsoft's internal conformer-encoder model. Here is an example of how to run the ORT transformer optimizer for the SigLIP model. ``` $ git clone https://github.com/microsoft/onnxruntime $ cd onnxruntime/onnxruntime/python/tools/transformers $ python3 optimizer.py --input /path/to/model.onnx --output /path/to/model_opt.onnx --model_type clip --num_heads 16 --hidden_size 1152 --use_external_data_format --opt_level 0 --disable_shape_inference ``` Here is an example of how to run the ORT transformer optimizer for the conformer-encoder model. ``` $ git clone https://github.com/microsoft/onnxruntime $ cd onnxruntime/onnxruntime/python/tools/transformers $ python3 optimizer.py --input /path/to/model.onnx --output /path/to/model_opt.onnx --model_type conformer --num_heads 16 --hidden_size 1024 --use_external_data_format --opt_level 0 --disable_shape_inference --convert_attribute ``` ### Motivation and Context This PR helps optimize multi-modal models that use SigLIP for the vision encoder and conformer-encoder for the speech encoder. This PR uses changes from the following PRs: - pytorch/pytorch#144801 - microsoft/onnxscript#2018 - microsoft/onnxscript#2019 - microsoft/onnxscript#2020 - microsoft/onnxscript#2021 - microsoft/onnxscript#2022 - microsoft/onnxscript#2024 - microsoft/onnxscript#2025 - microsoft/onnxscript#2029 - microsoft/onnxscript#2033 ### Introduction of ONNX Script This PR introduces [ONNX Script](https://github.com/microsoft/onnxscript) into the ORT transformer optimizer as an optional step via the `fold_transpose_initializers()` method of the `DynamoOnnxHelper` class.

justinchuby added 2 commits January 18, 2025 10:22

[IR] Create predecessors() and successors() on ir.Node

4c84dcf

lint

8e80f58

justinchuby added the module: IR Intermediate representation label Jan 18, 2025

justinchuby requested review from Copilot and titaiwangms January 18, 2025 18:49

Copilot AI reviewed Jan 18, 2025

View reviewed changes

justinchuby commented Jan 18, 2025

View reviewed changes

onnxscript/ir/_core.py Outdated Show resolved Hide resolved

Update onnxscript/ir/_core.py

3ae7882

github-advanced-security bot found potential problems Jan 18, 2025

View reviewed changes

onnxscript/ir/_core.py Fixed Show fixed Hide fixed

update

1793196

github-advanced-security bot found potential problems Jan 18, 2025

View reviewed changes

onnxscript/ir/_core.py Fixed Show fixed Hide fixed

onnxscript/ir/_core.py Fixed Show fixed Hide fixed

justinchuby added 5 commits January 18, 2025 17:21

test

7713268

uses

c30fc53

Add consumers

ca7c1f9

type

54348e1

usage

e208989

justinchuby requested a review from Copilot January 19, 2025 03:23

Copilot AI reviewed Jan 19, 2025

View reviewed changes

onnxscript/ir/_core.py Outdated Show resolved Hide resolved

onnxscript/ir/_core.py Outdated Show resolved Hide resolved

justinchuby and others added 2 commits January 18, 2025 19:31

Update onnxscript/ir/_core.py

a1f1de6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update onnxscript/ir/_core.py

c7d16c5

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

justinchuby commented Jan 19, 2025

View reviewed changes

onnxscript/ir/_core.py Outdated Show resolved Hide resolved

justinchuby added 3 commits January 18, 2025 19:32

Update onnxscript/ir/_core.py

85e1585

test

4bcfc74

test

b9f76cc

justinchuby requested review from Copilot and gramalingam January 19, 2025 03:45

Copilot AI reviewed Jan 19, 2025

View reviewed changes

onnxscript/ir/_core.py Show resolved Hide resolved

onnxscript/ir/_core.py Show resolved Hide resolved

onnxscript/ir/_core.py Show resolved Hide resolved

test_predecessors_are_unique

185b64d

justinchuby commented Jan 21, 2025

View reviewed changes

onnxscript/ir/_core.py Show resolved Hide resolved

justinchuby commented Jan 21, 2025

View reviewed changes

onnxscript/ir/_core.py Show resolved Hide resolved

justinchuby commented Jan 21, 2025

View reviewed changes

onnxscript/ir/_core.py Show resolved Hide resolved

titaiwangms approved these changes Jan 21, 2025

View reviewed changes

gramalingam reviewed Jan 21, 2025

View reviewed changes

onnxscript/ir/_core.py Show resolved Hide resolved

justinchuby merged commit 969c078 into main Jan 21, 2025
24 of 29 checks passed

justinchuby deleted the justinchu/node-predecessors branch January 21, 2025 17:07

kunal-vaishnavi mentioned this pull request Jan 29, 2025

Add fusions for SigLIP and Conformer-Encoder microsoft/onnxruntime#23528

Merged

[IR] Create predecessors() and successors() on ir.Node #2022

[IR] Create predecessors() and successors() on ir.Node #2022

Uh oh!

Conversation

justinchuby commented Jan 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jan 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 70 Tests Failed:

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

justinchuby commented Jan 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[IR] Create `predecessors()` and `successors()` on `ir.Node` #2022

[IR] Create `predecessors()` and `successors()` on `ir.Node` #2022

justinchuby commented Jan 18, 2025 •

edited

Loading

codecov bot commented Jan 18, 2025 •

edited

Loading