[Inductor] Pick ISA for inductor based on ATEN_CPU_CAPABILITY by CaoE · Pull Request #123514 · pytorch/pytorch

CaoE · 2024-04-07T03:26:15Z

Stack from ghstack (oldest at bottom):

-> [Inductor] Pick ISA for inductor based on ATEN_CPU_CAPABILITY #123514

It is part of #123224. Pick ISA based on the environment ATEN_CPU_CAPABILITY to control CPU vec ISA level for Inductor like eager.

cc @ezyang @chauhang @penguinwu @voznesenskym @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @rec @msaroufim @bdhirsh @anijain2305 @peterbell10 @aakhundov

[ghstack-poisoned]

pytorch-bot · 2024-04-07T03:26:19Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/123514

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 38ec5c4 with merge base 67883e7 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

periodic / win-vs2019-cuda11.8-py3 / test (default, 1, 4, windows.g5.4xlarge.nvidia.gpu) (gh) (disabled by #137936)
test_linalg.py::TestLinalgCUDA::test_matmul_offline_tunableop_cuda_float16

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]

ghstack-source-id: 75ae45b Pull Request resolved: #123514

jgong5 · 2024-04-08T01:44:09Z

torch/_inductor/codecache.py

+        and not isinstance(_valid_vec_isa_list[0], VecNEON)
+        and not isinstance(_valid_vec_isa_list[0], VecZVECTOR)


compute_cpu_capability also handles vsx and zvector. We should also consider them here?

Taking VecNEON and VecZVECTOR into account.

jgong5 · 2024-04-08T01:47:50Z

aten/src/ATen/native/DispatchStub.cpp


 CPUCapability get_cpu_capability() {
-  static CPUCapability capability = compute_cpu_capability();
+  CPUCapability capability = compute_cpu_capability();


Why removing static here?

Changed it back

jgong5 · 2024-04-08T01:49:10Z

torch/_inductor/codecache.py

    # If the simdlen is None, it indicates determin the vectorization length automatically
    if config.cpp.simdlen is None:
        assert _valid_vec_isa_list


perhaps we don't need this check any longer.

Removed the check

[ghstack-poisoned]

ghstack-source-id: 8b0e45e Pull Request resolved: #123514

CaoE · 2024-06-25T02:43:27Z

This PR depends on #124245

github-actions · 2024-08-24T03:34:01Z

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[ghstack-poisoned]

jgong5

Seems a lot of UT failures. Can you check?

[ghstack-poisoned]

jgong5 · 2024-09-25T06:16:31Z

test/inductor/test_torchinductor.py

-            atol=atol,
-            rtol=rtol,
+            atol=1e-3,
+            rtol=1e-3,


This changes the tolerance for all devices. Is it expected?

I just changed this back to how it was before in this PR, no new changes were introduced.

CaoE · 2024-09-27T01:13:05Z

Hi @jgong5
Here is a new thing: Inductor recently added AMX ISA level which inherits AVX512.

class VecAMX(VecAVX512):
    _arch_flags = VecAVX512._arch_flags + " -mamx-tile -mamx-bf16 -mamx-int8"
    def __str__(self) -> str:
        return super().__str__() + " amx_tile"
    __hash__: Callable[[VecISA], Any] = VecISA.__hash__
...
supported_vec_isa_list = [VecAMX(), VecAVX512(), VecAVX2(), VecNEON()]

Eager does not have the option to set the AMX ISA level. When ATEN_CPU_CAPABILITY is set to avx512, both VecAMX and VecAVX512 of Inductor satisfy this environment variable, and VecAMX will be prioritized for selection.

CaoE · 2024-09-30T00:46:10Z

@pytorchbot merge

pytorchmergebot · 2024-09-30T00:47:47Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

huydhn · 2024-09-30T15:45:14Z

@pytorchbot revert -m 'Sorry for reverting your change but its test_cpu_repro test is failing in trunk https://hud.pytorch.org/pytorch/pytorch/commit/6931c1644afdba53e63ce5671455e4e1b7265dd9' -c nosignal

I think a rebase is needed

pytorchmergebot · 2024-09-30T15:46:54Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2024-09-30T15:47:07Z

@CaoE your PR has been successfully reverted.

[ghstack-poisoned]

github-actions

Please commit the suggested changes from pytorch's linter.

test/inductor/test_cpu_repro.py

[ghstack-poisoned]

CaoE · 2024-10-13T11:24:18Z

@huydhn The PR is rebased. Could you please help import this this PR to see if it will break internal checks ?

huydhn · 2024-10-14T10:17:28Z

Umm, unfortunately, I couldn't import this PR because it uses ghstack. Only the stack owner can do so (folks from Meta do that themselves). There is no work around that I know atm, so I think let's just merge this and let our oncall check it later then.

CaoE · 2024-10-17T08:58:55Z

@pytorchbot merge

pytorchmergebot · 2024-10-17T09:01:29Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Update

3996e44

[ghstack-poisoned]

pytorch-bot bot added ciflow/inductor module: inductor labels Apr 7, 2024

CaoE marked this pull request as draft April 7, 2024 03:26

CaoE changed the title ~~Set sisdlen according to _get_cpu_capability~~ Set simdlen according to _get_cpu_capability Apr 7, 2024

CaoE added ciflow/trunk Trigger trunk jobs on your pull request ciflow/periodic Trigger jobs ran periodically on master (periodic.yml) on the PR labels Apr 7, 2024

pytorchbot added the open source label Apr 7, 2024

Update

6da1626

[ghstack-poisoned]

Update

ab7dae3

[ghstack-poisoned]

CaoE added a commit that referenced this pull request Apr 7, 2024

Set simdlen according to _get_cpu_capability

3a172d9

ghstack-source-id: 75ae45b Pull Request resolved: #123514

CaoE requested a review from jgong5 April 7, 2024 12:09

jgong5 requested changes Apr 8, 2024

View reviewed changes

Update

4d64aef

[ghstack-poisoned]

Update

8313e6a

[ghstack-poisoned]

Update

1ab6aa5

[ghstack-poisoned]

Update

172a07a

[ghstack-poisoned]

Update

14d97dc

[ghstack-poisoned]

CaoE added a commit that referenced this pull request Apr 9, 2024

Set simdlen according to _get_cpu_capability

9effc41

ghstack-source-id: 8b0e45e Pull Request resolved: #123514

CaoE changed the title ~~Set simdlen according to _get_cpu_capability~~ Set simdlen based on the environment ATEN_CPU_CAPABILITY Apr 9, 2024

CaoE changed the title ~~Set simdlen based on the environment ATEN_CPU_CAPABILITY~~ Set simdlen based on ATEN_CPU_CAPABILITY Apr 9, 2024

This was referenced Jun 12, 2024

Revert "Set simdlen based on ATEN_CPU_CAPABILITY (#123514)" #128541

Merged

[v.2.4.0] Release Tracker #128436

Closed

leslie-fang-intel mentioned this pull request Aug 22, 2024

[CI] CPU Inductor codepath for AVX2/Default is not tested in CI #123224

Closed

Update

05fb052

[ghstack-poisoned]

jgong5 requested changes Sep 15, 2024

View reviewed changes

CaoE added 6 commits September 19, 2024 01:31

Update

5c5d4a2

[ghstack-poisoned]

Update

383db18

[ghstack-poisoned]

Update

047e3bd

[ghstack-poisoned]

Update

ba970c2

[ghstack-poisoned]

Update

fc527f4

[ghstack-poisoned]

Update

33054cc

[ghstack-poisoned]

jgong5 approved these changes Sep 25, 2024

View reviewed changes

abhishek-iitmadras mentioned this pull request Sep 30, 2024

Extend vectorization with SVE(ARM) with Torch Compile (Inductor) #134672

Closed

CaoE added 2 commits October 8, 2024 23:53

Update

53d3291

[ghstack-poisoned]

Update

b5c77c0

[ghstack-poisoned]

github-actions bot requested changes Oct 11, 2024

View reviewed changes

test/inductor/test_cpu_repro.py Show resolved Hide resolved

CaoE added 2 commits October 10, 2024 23:57

Update

085b4cd

[ghstack-poisoned]

Update

38ec5c4

[ghstack-poisoned]

		and not isinstance(_valid_vec_isa_list[0], VecNEON)
		and not isinstance(_valid_vec_isa_list[0], VecZVECTOR)

Conversation

CaoE commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/123514

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

jgong5 Apr 8, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE Apr 12, 2024

Choose a reason for hiding this comment

Uh oh!

jgong5 Apr 8, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE Apr 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgong5 Apr 8, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE Apr 12, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE commented Jun 25, 2024

Uh oh!

github-actions bot commented Aug 24, 2024

Uh oh!

jgong5 left a comment

Choose a reason for hiding this comment

Uh oh!

jgong5 Sep 25, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE Sep 27, 2024

Choose a reason for hiding this comment

Uh oh!

CaoE commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CaoE commented Sep 30, 2024

Uh oh!

pytorchmergebot commented Sep 30, 2024

Merge started

Uh oh!

huydhn commented Sep 30, 2024

Uh oh!

pytorchmergebot commented Sep 30, 2024

Uh oh!

pytorchmergebot commented Sep 30, 2024

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CaoE commented Oct 13, 2024

Uh oh!

huydhn commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CaoE commented Oct 17, 2024

Uh oh!

pytorchmergebot commented Oct 17, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

CaoE commented Apr 7, 2024 •

edited

Loading

pytorch-bot bot commented Apr 7, 2024 •

edited

Loading

CaoE Apr 12, 2024 •

edited

Loading

CaoE commented Sep 27, 2024 •

edited

Loading

huydhn commented Oct 14, 2024 •

edited

Loading