-
Notifications
You must be signed in to change notification settings - Fork 27.3k
Changing as_strided_scatter to deterministic inputs #85583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
1c8f4c1
WPI: changing as_strided_scatter to deterministic inputs
srossross 839ffa2
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 904fd95
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 63781d3
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross b735e96
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 853b9fe
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 73497a2
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 46036da
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross ea3ea66
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 7df7c65
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross d23afcf
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 9da3db3
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 2320d5e
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 9c10a5f
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 8cc7a77
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross a12555d
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 3add5fd
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 2c7f50e
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 28aba99
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 068108f
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 5cbff47
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross dfc65be
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 6787c1d
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 2906c8a
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross 854d431
Update on "WIP: changing as_strided_scatter to deterministic inputs"
srossross f5f039e
Update on "Changing as_strided_scatter to deterministic inputs"
srossross 4a9ef27
Update on "Changing as_strided_scatter to deterministic inputs"
srossross 4b64fe2
Update on "Changing as_strided_scatter to deterministic inputs"
srossross 0b103b0
Update on "Changing as_strided_scatter to deterministic inputs"
srossross File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -274,15 +274,30 @@ def sample_inputs_as_strided_scatter(op_info, device, dtype, requires_grad, **kw | |
| ((1,), (1,), (1,), 0), | ||
| ((3, 3), (2, 2), (1, 2), 0), | ||
| ((3, 3), (2, 2), (1, 2), 1), | ||
| ((16,), (2, 2, 2, 2), (1, 1, 1, 1), 0), | ||
| ((16,), (2, 1, 1, 2), (1, 7, 7, 1), 0), | ||
| ((3, 3), (2, 2), (2, 1), 0), | ||
| # Scatter to larger dimentions | ||
| ((16,), (2, 2, 2, 2), (8, 4, 2, 1), 0), | ||
| # Scatter to larger dimentions with strides inverted | ||
| ((16,), (2, 1, 1, 2), (1, 2, 4, 8), 0), | ||
| ] | ||
|
|
||
| for input_shape, output_shape, stride, storage_offset in test_cases: | ||
| input_t = make_arg(input_shape) | ||
| input_src = make_arg(output_shape) | ||
| yield SampleInput(input_t, input_src, output_shape, stride, storage_offset=storage_offset) | ||
|
|
||
| def error_inputs_as_strided_scatter(op_info, device, **kwargs): | ||
| make_arg = partial(make_tensor, device=device, dtype=torch.float32, requires_grad=False) | ||
|
|
||
| # Create a small tensor and try to scatter it out of bounds | ||
| input_t = make_arg([4, 4]) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Add a comment describing what error condition is being tested
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. added |
||
| input_src = make_arg([2, 2]) | ||
| yield ErrorInput( | ||
| SampleInput(input_t, input_src, [2, 2], [200, 200], storage_offset=0), | ||
| error_regex="itemsize 4 requiring a storage size of 1604 are out of bounds for storage of size 64" | ||
| ) | ||
|
|
||
|
|
||
| def sample_inputs_combinations(op_info, device, dtype, requires_grad, **kwargs): | ||
| inputs = ( | ||
| (0,), | ||
|
|
@@ -10430,18 +10445,15 @@ def reference_flatten(input, start_dim=0, end_dim=-1): | |
| # vmap does not support inplace views | ||
| check_inplace_batched_forward_grad=False, | ||
| sample_inputs_func=sample_inputs_as_strided_scatter, | ||
| error_inputs_func=error_inputs_as_strided_scatter, | ||
| skips=( | ||
| DecorateInfo(unittest.skip('Works for int64, fails for everything else'), 'TestCommon', 'test_noncontiguous_samples'), # noqa: B950 | ||
| DecorateInfo(unittest.skip('Fails in most cases, passes on LAZY for some reason'), 'TestCommon', 'test_variant_consistency_eager'), # noqa: B950 | ||
| DecorateInfo(unittest.skip('Fails on cuda + rocm'), 'TestCommon', 'test_complex_half_reference_testing'), | ||
| DecorateInfo(unittest.expectedFailure, 'TestBwdGradients', 'test_fn_grad'), | ||
| DecorateInfo(unittest.expectedFailure, 'TestBwdGradients', 'test_fn_gradgrad'), | ||
| DecorateInfo(unittest.skip('Passes on complex128 and float64 only'), 'TestFwdGradients', 'test_fn_fwgrad_bwgrad'), | ||
| # AssertionError: Tensor-likes are not close! (new_empty_strided.default) | ||
| DecorateInfo(unittest.skip("Expected: new_empty_strided is not comparable"), 'TestDecomp', 'test_comprehensive'), | ||
| DecorateInfo( | ||
| unittest.skip("Some stride values write multiple values to the same location e.g. (1,1,1,1)"), | ||
| 'TestCommon', 'test_compare_cpu'),)), | ||
| DecorateInfo(unittest.skip("Expected: new_empty_strided is not comparable"), 'TestDecomp', 'test_comprehensive'),)), | ||
| OpInfo('native_layer_norm', | ||
| aten_name='native_layer_norm', | ||
| ref=reference_native_layer_norm, | ||
|
|
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the change here? Should the test cases have a comment describing the property they assume?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added a comment above, the removed code was causing non-deterministic behaviour so I modified the strides