🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/142160

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 763b7e4 with merge base 0318589 ():

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorchbot label "release notes: cuda"

FYI: @albanD, reopened #142090

cc: @miladm for visibility, since this is somewhat an extension/related to pytorch/xla#8450.

@eqy can you take a look at this?

@albanD, @eqy Any way we can consider this in for 2.6? We need it for our Torch XLA counterpart PRs.

✅ Deploy Preview for chimerical-cranachan-793287 ready!

Name	Link
🔨 Latest commit	763b7e4
🔍 Latest deploy log	https://app.netlify.com/sites/chimerical-cranachan-793287/deploys/67539246757645000865b5a8
😎 Deploy Preview	https://deploy-preview-142160--chimerical-cranachan-793287.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Fixed a typo in the CUDA file, PTAL.

@eqy Can you help restart the WF? Thank you.

Would it make sense to add some trivial asserts that guarantee that the number of elements of the tensor doesn't exceed the max possible value of the provided index type?

The indices are for the elements along a specific dimension, and these are implicitly capped by the index type. The total number of elements for the indices can have repeated values if I am not mistaken, so the assertion wouldn't necessarily hold. I also thought of aligning the first dimension type with the indices, but doesn't necessarily need to hold, but we do have an assert for that:

          TORCH_CHECK_INDEX(idx >= -self_dim_size && idx < self_dim_size,
                            "index ", idx, " is out of bounds for dimension ",
                            dim, " with size ", self_dim_size);

This run seems flaky https://github.com/pytorch/pytorch/actions/runs/12208028139/job/34061080338?pr=142160#step:7:15939.

Would it make sense to add some trivial asserts that guarantee that the number of elements of the tensor doesn't exceed the max possible value of the provided index type?

The indices are for the elements along a specific dimension, and these are implicitly capped by the index type. The total number of elements for the indices can have repeated values if I am not mistaken, so the assertion wouldn't necessarily hold. I also thought of aligning the first dimension type with the indices, but doesn't necessarily need to hold, but we do have an assert for that:

          TORCH_CHECK_INDEX(idx >= -self_dim_size && idx < self_dim_size,
                            "index ", idx, " is out of bounds for dimension ",
                            dim, " with size ", self_dim_size);

What I mean is, does it make sense to add a check against a dim with size > INT_MAX when 32-bit indices are used?

Would it make sense to add some trivial asserts that guarantee that the number of elements of the tensor doesn't exceed the max possible value of the provided index type?

The indices are for the elements along a specific dimension, and these are implicitly capped by the index type. The total number of elements for the indices can have repeated values if I am not mistaken, so the assertion wouldn't necessarily hold. I also thought of aligning the first dimension type with the indices, but doesn't necessarily need to hold, but we do have an assert for that:

          TORCH_CHECK_INDEX(idx >= -self_dim_size && idx < self_dim_size,
                            "index ", idx, " is out of bounds for dimension ",
                            dim, " with size ", self_dim_size);

What I mean is, does it make sense to add a check against a dim with size > INT_MAX when 32-bit indices are used?

I see, thanks for your suggestion - and if I am not mistaken, this already is in place. In the current kernel implementation, when we iterate through the indices, the existing TORCH_CHECK_INDEX macro should already do this bounds checking for each index against dim (size > idx <= INT_MAX, for int32). This should already make it a safe operation for handling 32-bit indices: https://github.com/pytorch/pytorch/pull/142160/files#diff-8aa1a200ec63d23db422aa31b6dca1e6cb372887c43b064ef435210b1b0dec0aR3446. If you instead mean the dim input, then these represent the dim tensor that we operate within, it wouldn't be directly related. Let me know if this helps answer it.

@eqy do you have other concerns here? Let me know if any, in case this could still go in for 2.6.

My concern was not memory safety, but rather warning/alerting the user in cases where the index width could not be used to fully address a given dimension, e.g., dim size > 2**32 but index type int. In other words, does the "implict capping" lead to potentially nonsensical use-cases that should raise a warning or exception?

A trivial code example:

>>> import torch
>>> a = torch.empty(2**32, device='cuda', dtype=torch.uint8)
>>> a.index_fill_(0, torch.tensor([2**32 - 1], dtype=torch.int64, device='cuda'), -1)
tensor([  0,   0,   0,  ...,   0,   0, 255], device='cuda:0',
       dtype=torch.uint8)
>>> a.index_fill_(0, torch.tensor([2**32 - 1], dtype=torch.int64, device='cuda').to(torch.int32), -1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: index_fill_(): Expected dtype int64 for index.

What would the second case do here? IMO relying on potential UB (e.g., assuming the conversion rolls negative) seems flaky

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

@pytorchbot label "no-stale"

Keep this alive.

See #141994 for a related RFC.

My concern was not memory safety, but rather warning/alerting the user in cases where the index width could not be used to fully address a given dimension, e.g., dim size > 2**32 but index type int. In other words, does the "implict capping" lead to potentially nonsensical use-cases that should raise a warning or exception?

#142160 (comment)

@rpsilva-aws @eqy can the following be used to check for index bound?

canUse32BitIndexMath

can_use_32bit_indexing

@rpsilva-aws will you rebase?