Overload _get_operation_for_overload_or_packet & friends to accept ArrayRef by swolchok · Pull Request #162219 · pytorch/pytorch

swolchok · 2025-09-04T22:25:18Z

Stack from ghstack (oldest at bottom):

Avoids requiring vector allocation to call this.

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel

…rayRef Avoids requiring vector allocation to call this. [ghstack-poisoned]

pytorch-bot · 2025-09-04T22:25:21Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162219

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d5990af with merge base 2b8a839 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Skylion007 · 2025-09-05T16:07:45Z

+                        op, symbol, args, kwargs, /*is_overload*/ true, dk_);
                  });
              return py::make_tuple(
                  func, func_dk, py::cast(op->getTags().vec()));


These values should be std::move to prevent ref count increases

I don't see anything movable here; if we move op from the lambda capture then the lambda won't work if called a second time.

This just moves the Python reference to the cpp_function pybind11 object. Isn't that reference created every time?

This just avoid a pointer inc / decrease of the Python reference counter, it shouldn't move the function inside of the Python function since it's effectively a shared ptr, right?

To clarify, suggesting moving func and func_dk

…o accept ArrayRef" Avoids requiring vector allocation to call this. cc EikanWang jgong5 wenzhe-nrv sanchitintel [ghstack-poisoned]

pytorchmergebot · 2025-09-08T22:30:15Z

Starting merge as part of PR stack under #162220

Per Skylion007 on #162219 ghstack-source-id: 00ea4c1 Pull-Request: #162428

Per Skylion007 on #162219 ghstack-source-id: de3fd28 Pull-Request: #162428

@Skylion007

Per @Skylion007 on #162219 Pull Request resolved: #162428 Approved by: https://github.com/Skylion007

…2218) It returns a const reference to a vector. Pull Request resolved: #162218 Approved by: https://github.com/Skylion007 ghstack dependencies: #161591, #161595, #161633, #161634, #161692, #162219, #162220

These seem to have been costing us 5-10 usec per detach (out of ~~95 usec total). If they need to ship let's talk about requirements and how we can make this more efficient given that we would prefer if an entire DTensor op could finish in 10 usec. Differential Revision: [D81530106](https://our.internmc.facebook.com/intern/diff/D81530106) Pull Request resolved: #161596 Approved by: https://github.com/ezyang, https://github.com/Skylion007 ghstack dependencies: #161591, #161595, #161633, #161634, #161692, #162219, #162220, #162218

We control DTensor, so we can just guarantee there isn't a programming error with __torch_dispatch__. (The guard is already less-than-perfect; see the note that the deleted comment refers to.) Pull Request resolved: #162337 Approved by: https://github.com/Skylion007 ghstack dependencies: #161591, #161595, #161633, #161634, #161692, #162219, #162220, #162218, #161596

…rayRef (pytorch#162219) Avoids requiring vector allocation to call this. Pull Request resolved: pytorch#162219 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692

Optimize for common case and remove a pair of refcount operations (see new comments.) Pull Request resolved: pytorch#162220 Approved by: https://github.com/jansel, https://github.com/williamwen42 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219

@Skylion007

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

…orch#162218) It returns a const reference to a vector. Pull Request resolved: pytorch#162218 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220

These seem to have been costing us 5-10 usec per detach (out of ~~95 usec total). If they need to ship let's talk about requirements and how we can make this more efficient given that we would prefer if an entire DTensor op could finish in 10 usec. Differential Revision: [D81530106](https://our.internmc.facebook.com/intern/diff/D81530106) Pull Request resolved: pytorch#161596 Approved by: https://github.com/ezyang, https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218

…162337) We control DTensor, so we can just guarantee there isn't a programming error with __torch_dispatch__. (The guard is already less-than-perfect; see the note that the deleted comment refers to.) Pull Request resolved: pytorch#162337 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218, pytorch#161596

…rayRef (pytorch#162219) Avoids requiring vector allocation to call this. Pull Request resolved: pytorch#162219 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692

Optimize for common case and remove a pair of refcount operations (see new comments.) Pull Request resolved: pytorch#162220 Approved by: https://github.com/jansel, https://github.com/williamwen42 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219

@Skylion007

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

…orch#162218) It returns a const reference to a vector. Pull Request resolved: pytorch#162218 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220

These seem to have been costing us 5-10 usec per detach (out of ~~95 usec total). If they need to ship let's talk about requirements and how we can make this more efficient given that we would prefer if an entire DTensor op could finish in 10 usec. Differential Revision: [D81530106](https://our.internmc.facebook.com/intern/diff/D81530106) Pull Request resolved: pytorch#161596 Approved by: https://github.com/ezyang, https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218

…162337) We control DTensor, so we can just guarantee there isn't a programming error with __torch_dispatch__. (The guard is already less-than-perfect; see the note that the deleted comment refers to.) Pull Request resolved: pytorch#162337 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218, pytorch#161596

…rayRef (pytorch#162219) Avoids requiring vector allocation to call this. Pull Request resolved: pytorch#162219 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692

Optimize for common case and remove a pair of refcount operations (see new comments.) Pull Request resolved: pytorch#162220 Approved by: https://github.com/jansel, https://github.com/williamwen42 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219

@Skylion007

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

…orch#162218) It returns a const reference to a vector. Pull Request resolved: pytorch#162218 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220

These seem to have been costing us 5-10 usec per detach (out of ~~95 usec total). If they need to ship let's talk about requirements and how we can make this more efficient given that we would prefer if an entire DTensor op could finish in 10 usec. Differential Revision: [D81530106](https://our.internmc.facebook.com/intern/diff/D81530106) Pull Request resolved: pytorch#161596 Approved by: https://github.com/ezyang, https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218

…162337) We control DTensor, so we can just guarantee there isn't a programming error with __torch_dispatch__. (The guard is already less-than-perfect; see the note that the deleted comment refers to.) Pull Request resolved: pytorch#162337 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218, pytorch#161596

…rayRef (pytorch#162219) Avoids requiring vector allocation to call this. Pull Request resolved: pytorch#162219 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692

Optimize for common case and remove a pair of refcount operations (see new comments.) Pull Request resolved: pytorch#162220 Approved by: https://github.com/jansel, https://github.com/williamwen42 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219

@Skylion007

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

…orch#162218) It returns a const reference to a vector. Pull Request resolved: pytorch#162218 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220

These seem to have been costing us 5-10 usec per detach (out of ~~95 usec total). If they need to ship let's talk about requirements and how we can make this more efficient given that we would prefer if an entire DTensor op could finish in 10 usec. Differential Revision: [D81530106](https://our.internmc.facebook.com/intern/diff/D81530106) Pull Request resolved: pytorch#161596 Approved by: https://github.com/ezyang, https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218

…162337) We control DTensor, so we can just guarantee there isn't a programming error with __torch_dispatch__. (The guard is already less-than-perfect; see the note that the deleted comment refers to.) Pull Request resolved: pytorch#162337 Approved by: https://github.com/Skylion007 ghstack dependencies: pytorch#161591, pytorch#161595, pytorch#161633, pytorch#161634, pytorch#161692, pytorch#162219, pytorch#162220, pytorch#162218, pytorch#161596

Overload _get_operation_for_overload_or_packet & friends to accept Ar…

0305719

…rayRef Avoids requiring vector allocation to call this. [ghstack-poisoned]

swolchok requested a review from mikaylagawarecki as a code owner September 4, 2025 22:25

pytorch-bot Bot added the release notes: jit release notes category label Sep 4, 2025

facebook-github-bot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Sep 4, 2025

Skylion007 reviewed Sep 5, 2025

View reviewed changes

Update on "Overload _get_operation_for_overload_or_packet & friends t…

144e44c

…o accept ArrayRef" Avoids requiring vector allocation to call this. cc EikanWang jgong5 wenzhe-nrv sanchitintel [ghstack-poisoned]

swolchok mentioned this pull request Sep 5, 2025

Add DISABLE_JUSTKNOBS to torch/_utils_internal.py and use it for dynamo _maybe_set_eval_frame #162298

Closed

swolchok requested a review from Skylion007 September 6, 2025 17:28

Update on "Overload _get_operation_for_overload_or_packet & friends t…

81fa133

…o accept ArrayRef" Avoids requiring vector allocation to call this. cc EikanWang jgong5 wenzhe-nrv sanchitintel [ghstack-poisoned]

This was referenced Sep 3, 2025

Port OpSchema.__post_init__ and OpSchema._recompute_comparison_key to C++ #161695

Closed

Fix TODO in make_tensor_for_subclass_helper #162336

Closed

Remove __torch_dispatch__ check in THPVariable_make_dtensor #162337

Closed

Skylion007 approved these changes Sep 6, 2025

View reviewed changes

Update on "Overload _get_operation_for_overload_or_packet & friends t…

d5990af

…o accept ArrayRef" Avoids requiring vector allocation to call this. cc EikanWang jgong5 wenzhe-nrv sanchitintel [ghstack-poisoned]

swolchok mentioned this pull request Sep 8, 2025

Fix missing moves in initJITBindings #162428

Closed

swolchok added a commit that referenced this pull request Sep 8, 2025

Fix missing moves in initJITBindings

240e96a

Per Skylion007 on #162219 ghstack-source-id: 00ea4c1 Pull-Request: #162428

swolchok added a commit that referenced this pull request Sep 8, 2025

Fix missing moves in initJITBindings

974fe18

Per Skylion007 on #162219 ghstack-source-id: de3fd28 Pull-Request: #162428

pytorchmergebot pushed a commit that referenced this pull request Sep 9, 2025

Fix missing moves in initJITBindings (#162428)

dcc42e9

Per @Skylion007 on #162219 Pull Request resolved: #162428 Approved by: https://github.com/Skylion007

swolchok mentioned this pull request Sep 9, 2025

Fully native DTensor.__new__ #162508

Closed

markc-614 pushed a commit to markc-614/pytorch that referenced this pull request Sep 17, 2025

Fix missing moves in initJITBindings (pytorch#162428)

dcf97a1

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

mansiag05 pushed a commit to mansiag05/pytorch that referenced this pull request Sep 22, 2025

Fix missing moves in initJITBindings (pytorch#162428)

21e4e3f

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

cleonard530 pushed a commit to cleonard530/pytorch that referenced this pull request Sep 22, 2025

Fix missing moves in initJITBindings (pytorch#162428)

209a964

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

dsashidh pushed a commit to dsashidh/pytorch that referenced this pull request Sep 26, 2025

Fix missing moves in initJITBindings (pytorch#162428)

6af531e

Per @Skylion007 on pytorch#162219 Pull Request resolved: pytorch#162428 Approved by: https://github.com/Skylion007

github-actions Bot deleted the gh/swolchok/827/head branch October 9, 2025 02:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Overload _get_operation_for_overload_or_packet & friends to accept ArrayRef#162219

Overload _get_operation_for_overload_or_packet & friends to accept ArrayRef#162219
swolchok wants to merge 4 commits intogh/swolchok/827/basefrom
gh/swolchok/827/head

swolchok commented Sep 4, 2025 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Sep 4, 2025 •

edited

Loading

Uh oh!

Skylion007 Sep 5, 2025

Uh oh!

swolchok Sep 6, 2025

Uh oh!

Skylion007 Sep 6, 2025

Uh oh!

Skylion007 Sep 6, 2025

Uh oh!

pytorchmergebot commented Sep 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

swolchok commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162219

✅ No Failures

Uh oh!

Skylion007 Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

Skylion007 Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

Skylion007 Sep 6, 2025

Choose a reason for hiding this comment

Uh oh!

pytorchmergebot commented Sep 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

swolchok commented Sep 4, 2025 •

edited

Loading

pytorch-bot Bot commented Sep 4, 2025 •

edited

Loading