Skip to content

Compute contiguity symbolically to avoid dde, and introduce c++ sym_is_contiguous#155590

Closed
laithsakka wants to merge 57 commits intogh/laithsakka/206/basefrom
gh/laithsakka/206/head
Closed

Compute contiguity symbolically to avoid dde, and introduce c++ sym_is_contiguous#155590
laithsakka wants to merge 57 commits intogh/laithsakka/206/basefrom
gh/laithsakka/206/head

Conversation

@laithsakka
Copy link
Contributor

@laithsakka laithsakka commented Jun 10, 2025

Stack from ghstack (oldest at bottom):

When we compute contiguity for a tensor with dynamic shapes we first:

  1. Try to compute it without guarding.
  2. If all shapes hinted, compute it with potentially adding guards.
  3. if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.

ex:
bool is_contiguous = input.sym_is_contiguous().guard_or_false(FILE, LINE);
is_contiguous_or_false is a helper function that does that.

In this PR I only handle default contiguity, will follow up with changes for other formats like channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: D77183032

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @jerryzh168 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Jun 10, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/155590

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Cancelled Jobs

As of commit 8c174c4 with merge base 0f9c1b3 (image):

NEW FAILURE - The following job has failed:

CANCELLED JOBS - The following jobs were cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ghstack-poisoned]
@laithsakka laithsakka changed the title to rebase [Draft] introduce c++ definitely_contiguous_fast. Jun 10, 2025
@laithsakka laithsakka added the topic: not user facing topic category label Jun 10, 2025
Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
@ColinPeppler
Copy link
Contributor

this one too

Tensor contiguous(const Tensor& self, MemoryFormat memory_format) {
if (self.is_contiguous(memory_format)) {

Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
@laithsakka
Copy link
Contributor Author

address failures add single unit test
one thing i want to test is that calling contiguous on a tensor with unbacked symbols followed by calling def_contig
should return True.

I think we might need to add the sym_max() in the def_contig to make it work like what i did in the python version,
but I will address that in its own PR and test it.

Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
@laithsakka laithsakka added the keep-going Don't stop on first failure, keep running tests until the end label Jun 11, 2025
Like is_contiguous, but more dynamic shape-friendly. Returns uncertain false instead of throwing 
data-dependent errors for tensors with unbacked sizes or strides that can be either contiguous or not.

The property is cached, meaning that if some runtime assertions are added that can be used to infer that the
 tensor is always contiguous after caching, this can still return false. 

This should be used in places where the contiguity check is not material or when there is a general path even
if less performant that  can be taken (for example, clone path in reshape or contiguous call).


[ghstack-poisoned]
…e c++ is_contiguous_or_false. "


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

is_contiguous_or_false is similar to is_contiguous but can return false instead of throwing DDE.
similarly,  is_non_overlapping_and_dense_or_false.

In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
This PR uses is_contiguous_or_false in contiguous, the semantics is if we do not know if the input is 
contiguous we just clone .

Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
…e c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
is_contiguous_or_false is a helper function that does that. 
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
@laithsakka
Copy link
Contributor Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team Raised by workflow job

@laithsakka
Copy link
Contributor Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@jeanschmidt
Copy link
Contributor

jeanschmidt commented Jul 1, 2025

@pytorchbot revert -m "Breaking 1000s of internal builds, it cant be properly landed internally, there are no options except revert and codev. D77556361" -c ghfirst

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

@laithsakka your PR has been successfully reverted.

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: Comment with id 3024775982 not found

Details for Dev Infra team Raised by workflow job

…de, and introduce c++ sym_is_contiguous"


When we compute contiguity for a tensor with dynamic shapes we first:
1) Try to compute it without guarding. 
2) If all shapes hinted, compute it with potentially adding guards. 
3) if any input is not hinted, compute it symbolically.

sym_is_contiguous return a SymBool that is then either evaluated or guard_or_false can be called
on it to avoid data dependent errors.  

ex:
 bool is_contiguous = input.sym_is_contiguous().guard_or_false(__FILE__, __LINE__);
is_contiguous_or_false is a helper function that does that. 
 
In this PR I only handle default contiguity, will follow up with changes for other formats like  channel_last .
We use this patter in this PR for several locations to avoid DDEs.
Differential Revision: [D77183032](https://our.internmc.facebook.com/intern/diff/D77183032)

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov

[ghstack-poisoned]
@laithsakka
Copy link
Contributor Author

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@laithsakka
Copy link
Contributor Author

laithsakka commented Jul 1, 2025

@pytorch revert -m "I was asked by infra team to land this internally first"

@laithsakka
Copy link
Contributor Author

@pytorchbot revert -m "was asked by to land this internally " -c ghfirst

@pytorchmergebot
Copy link
Collaborator

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

@pytorchmergebot
Copy link
Collaborator

@laithsakka your PR has been successfully reverted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-no-td Do not run TD on this PR ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request keep-going Don't stop on first failure, keep running tests until the end Merged module: cpu CPU specific problem (e.g., perf, algorithm) module: inductor Reverted topic: not user facing topic category

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants