Skip to content

Improve numerical stability of torch.distributions.wishart.Wishart#72059

Closed
nonconvexopt wants to merge 26 commits intopytorch:masterfrom
nonconvexopt:distributions/wishart_improvement_v1
Closed

Improve numerical stability of torch.distributions.wishart.Wishart#72059
nonconvexopt wants to merge 26 commits intopytorch:masterfrom
nonconvexopt:distributions/wishart_improvement_v1

Conversation

@nonconvexopt
Copy link
Copy Markdown
Contributor

@nonconvexopt nonconvexopt commented Jan 31, 2022

Maintanance of #70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc @neerajprad

Key modifications:

  • torch/distributions/wishart.py: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the eps value paired to each torch.dtype
  • test/distributions/test_distributions.py: Test Wishart distribution implementation in numerically unstable zones, i.e df values are at ndim - 1 < df < ndim where ndim is the one dimenstion of the Wishart parameter & sample matrix.

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Jan 31, 2022

CI Flow Status

⚛️ CI Flow

Ruleset - Version: v1
Ruleset - File: https://github.com/nonconvexopt/pytorch/blob/ec4366618d9ac34e26b451e48cceebf368dbe404/.github/generated-ciflow-ruleset.json
PR ciflow labels: ciflow/default
Add ciflow labels to this PR to trigger more builds:

Workflows Labels (bold enabled) Status
Triggered Workflows
linux-binary-conda ciflow/binaries, ciflow/binaries_conda, ciflow/default ✅ triggered
linux-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
linux-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
linux-binary-manywheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
linux-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/noarch, ciflow/trunk, ciflow/xla ✅ triggered
linux-docs ciflow/all, ciflow/cpu, ciflow/default, ciflow/docs, ciflow/linux, ciflow/trunk ✅ triggered
linux-vulkan-bionic-py3.7-clang9 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk, ciflow/vulkan ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-cuda11.3-py3.7-gcc7-bazel-test ciflow/all, ciflow/bazel, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-build ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3-clang5-mobile-custom-build-static ciflow/all, ciflow/default, ciflow/linux, ciflow/mobile, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-asan ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/sanitizers, ciflow/trunk ✅ triggered
linux-xenial-py3.7-clang7-onnx ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/onnx, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7 ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
linux-xenial-py3.7-gcc7-no-ops ciflow/all, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-gradle-custom-build-single-full-jit ciflow/all, ciflow/android, ciflow/cpu, ciflow/default, ciflow/linux, ciflow/trunk ✅ triggered
win-vs2019-cpu-py3 ciflow/all, ciflow/cpu, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
win-vs2019-cuda11.3-py3 ciflow/all, ciflow/cuda, ciflow/default, ciflow/trunk, ciflow/win ✅ triggered
windows-binary-libtorch-cxx11-abi ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
windows-binary-libtorch-pre-cxx11 ciflow/binaries, ciflow/binaries_libtorch, ciflow/default ✅ triggered
windows-binary-wheel ciflow/binaries, ciflow/binaries_wheel, ciflow/default ✅ triggered
Skipped Workflows
caffe2-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
docker-builds ciflow/all, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-custom-ops ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-arm64-metal ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64 ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-coreml ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
ios-12-5-1-x86-64-full-jit ciflow/all, ciflow/ios, ciflow/macos, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda10.2-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
libtorch-linux-xenial-cuda11.3-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/trunk 🚫 skipped
linux-bionic-cuda10.2-py3.9-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/slow, ciflow/trunk 🚫 skipped
linux-bionic-rocm4.5-py3.7 ciflow/linux, ciflow/rocm 🚫 skipped
linux-docs-push ciflow/all, ciflow/cpu, ciflow/linux, ciflow/scheduled 🚫 skipped
linux-xenial-cuda11.3-py3.7-gcc7-no-ops ciflow/all, ciflow/cuda, ciflow/linux, ciflow/trunk 🚫 skipped
macos-10-15-py3-arm64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-10-15-py3-lite-interpreter-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
macos-11-py3-x86-64 ciflow/all, ciflow/macos, ciflow/trunk 🚫 skipped
parallelnative-linux-xenial-py3.7-gcc5.4 ciflow/all, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped
periodic-libtorch-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-libtorch-linux-xenial-cuda11.1-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/libtorch, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-bionic-cuda11.5-py3.7-gcc7 ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-linux-xenial-cuda10.2-py3-gcc7-slow-gradcheck ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled, ciflow/slow, ciflow/slow-gradcheck 🚫 skipped
periodic-linux-xenial-cuda11.1-py3.7-gcc7-debug ciflow/all, ciflow/cuda, ciflow/linux, ciflow/scheduled 🚫 skipped
periodic-win-vs2019-cuda11.1-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
periodic-win-vs2019-cuda11.5-py3 ciflow/all, ciflow/cuda, ciflow/scheduled, ciflow/win 🚫 skipped
pytorch-linux-xenial-py3-clang5-android-ndk-r19c-build ciflow/all, ciflow/android, ciflow/cpu, ciflow/linux, ciflow/trunk 🚫 skipped

@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Jan 31, 2022

🔗 Helpful links

💊 CI failures summary and remediations

As of commit 4651368 (more details on the Dr. CI page):


💚 💚 Looks good so far! There are no failures yet. 💚 💚


This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

Comment thread torch/distributions/wishart.py
nonconvexopt added a commit to nonconvexopt/pytorch that referenced this pull request Jan 31, 2022
Copy link
Copy Markdown
Contributor

@neerajprad neerajprad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests are failing. Note that you should be able to run your test locally using:

pytest test/distributions/test_distributions.py -k {name of test}

You can build locally with pytest setup.py build. See this for details.


# Implemented Sampling using Bartlett decomposition
noise = self._dist_chi2.rsample(sample_shape).sqrt().diag_embed(dim1=-2, dim2=-1)
noise = _clamp_with_eps(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is the important change in this PR. Since there are some unrelated changes as well, could you add the specific change for better numerical stability in the PR description so that it is available for future reference?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the feedback. That would be great. I will summarize the Key modifications in the PR description.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that Chi2 (inherits from Gamma) already clamps based on .tiny. See

value.detach().clamp_(min=torch.finfo(value.dtype).tiny) # do not record in autograd graph
. I suppose this is adding some jitter to the diagonal elements, but is this high enough (I have seen 1e-6 or 1e-8 for float/double)?

Also, can we detach like in the example above so that its not recorded in autograd?

Copy link
Copy Markdown
Contributor Author

@nonconvexopt nonconvexopt Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose this is adding some jitter to the diagonal elements, but is this high enough (I have seen 1e-6 or 1e-8 for float/double)?

Thank you for the detailed points. I understand that it is not a neat way.
As you know, if we calculate reciprocal of float32 value clamped with tiny=1.17549e-38, we get numbers upper bounded by the exponent e^38 in base 10. But we get the number upper bounded by exponent e^7 if we clamp with eps=1e-7. Thus, I thought we need to clamp the value again to provide more stable code. Higher value might be better. I just adapted the information of the torch.finfo to make implementation to be seemed more principled.

Copy link
Copy Markdown
Contributor Author

@nonconvexopt nonconvexopt Feb 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can we detach like in the example above so that its not recorded in autograd?

I will detach it if you think it is needed. But, I don't understand the intuition. Doesn't it affect .rsample()? I will check it out, too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this will be used downstream so we shouldn't detach! Sorry about my hasty and misleading comment.

Copy link
Copy Markdown
Contributor Author

@nonconvexopt nonconvexopt Feb 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation. I think you found a potential issue in torch.distributions.gamma.Gamma. I wonder why there is a .detach either! Maybe we can fix it.

@nonconvexopt
Copy link
Copy Markdown
Contributor Author

You can build locally with pytest setup.py build. See this for details.

I will try to utilize my local environment. I appreciate your kindness.

@ngimel ngimel added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Feb 2, 2022
@facebook-github-bot
Copy link
Copy Markdown
Contributor

@neerajprad has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@nonconvexopt
Copy link
Copy Markdown
Contributor Author

Thanks for the fix and more robust testing.

It is my pleasure. Thank you for the review.

facebook-github-bot pushed a commit that referenced this pull request Feb 16, 2022
…72059)

Summary:
Maintanance of #70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: #72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
@github-actions
Copy link
Copy Markdown
Contributor

Hey @nonconvexopt.
You've committed this PR, but it does not have both a 'release notes: ...' and 'topics: ...' label. Please add one of each to the PR. The 'release notes: ...' label should represent the part of PyTorch that this PR changes (fx, autograd, distributed, etc) and the 'topics: ...' label should represent the kind of PR it is (not user facing, new feature, bug fix, perf improvement, etc). The list of valid labels can be found here for the 'release notes: ...' and here for the 'topics: ...'.
For changes that are 'topic: not user facing' there is no need for a release notes label.

@malfet
Copy link
Copy Markdown
Contributor

malfet commented Feb 16, 2022

Reverted this PR as it broke MacOS CI, see https://github.com/pytorch/pytorch/runs/5221143435?check_suite_focus=true

@facebook-github-bot
Copy link
Copy Markdown
Contributor

This pull request has been reverted by 0b117a3. To re-land this change, please open another pull request, assignthe same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).

@neerajprad
Copy link
Copy Markdown
Contributor

Here's the test failure for convenience:

======================================================================
FAIL [0.132s]: test_wishart_log_prob (__main__.TestDistributions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "distributions/test_distributions.py", line 2321, in test_wishart_log_prob
    self.assertEqual(0.0, (batched_prob - unbatched_prob).abs().max(), atol=1e-3, rtol=0)
  File "/usr/local/miniconda/envs/build/lib/python3.8/site-packages/torch/testing/_internal/common_utils.py", line 2121, in assertEqual
    assert_equal(
  File "/usr/local/miniconda/envs/build/lib/python3.8/site-packages/torch/testing/_comparison.py", line 1081, in assert_equal
    raise error_metas[0].to_error()
AssertionError: Scalars are not close!

@malfet: Are these tests run after merging? I see that the CI was green on this PR, and I want to better understand how to check for issues like this before I merge.

@malfet
Copy link
Copy Markdown
Contributor

malfet commented Feb 16, 2022

@neerajprad due to the limited capacity, MacOS tests are not run on PRs by default, one can opt in by adding ciflow/macos label

@nonconvexopt
Copy link
Copy Markdown
Contributor Author

I will try to fix the issue.

cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 17, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 20, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 20, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 20, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
cyyever pushed a commit to cyyever/pytorch_private that referenced this pull request Feb 21, 2022
…(#72059)

Summary:
Maintanance of pytorch/pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch/pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3ba465247f5777c3c40a90b96955c4281d0)
@facebook-github-bot
Copy link
Copy Markdown
Contributor

This pull request has been reverted by 0b117a3. To re-land this change, please open another pull request, assignthe same reviewers, fix the CI failures that caused the revert and make sure that the failing CI runs on the PR by applying the proper ciflow label (e.g., ciflow/trunk).

facebook-github-bot pushed a commit that referenced this pull request Mar 15, 2022
…72993)

Summary:
Maintanance of #70377
Multiple modifications of the merged initial implementation of Wishart distribution.

Key modifications:
* torch/distributions/wishart.py: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the eps value paired to each torch.dtype
* test/distributions/test_distributions.py: Test Wishart distribution implementation in numerically unstable zones, i.e df values are at ndim - 1 < df < ndim where ndim is the one dimenstion of the Wishart parameter & sample matrix.

Re-opened reverted PR #72059
cc neerajprad vadimkantorov

Pull Request resolved: #72993

Reviewed By: samdow

Differential Revision: D34853807

Pulled By: neerajprad

fbshipit-source-id: eb62dca19bf8a934fdf59b4ffc58587447fe8378
pytorchmergebot pushed a commit that referenced this pull request Mar 15, 2022
…72993)

Summary:
Maintanance of #70377
Multiple modifications of the merged initial implementation of Wishart distribution.

Key modifications:
* torch/distributions/wishart.py: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the eps value paired to each torch.dtype
* test/distributions/test_distributions.py: Test Wishart distribution implementation in numerically unstable zones, i.e df values are at ndim - 1 < df < ndim where ndim is the one dimenstion of the Wishart parameter & sample matrix.

Re-opened reverted PR #72059
cc neerajprad vadimkantorov

Pull Request resolved: #72993

Reviewed By: samdow

Differential Revision: D34853807

Pulled By: neerajprad

fbshipit-source-id: eb62dca19bf8a934fdf59b4ffc58587447fe8378
(cherry picked from commit 99240c0)
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
…ytorch#72059)

Summary:
Maintanance of pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.
cc neerajprad

Key modifications:
- `torch/distributions/wishart.py`: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the `eps` value paired to each `torch.dtype`
- `test/distributions/test_distributions.py`: Test Wishart distribution implementation in numerically unstable zones, i.e `df` values are at `ndim - 1 < df < ndim` where `ndim` is the one dimenstion of the Wishart parameter & sample matrix.

Pull Request resolved: pytorch#72059

Reviewed By: H-Huang

Differential Revision: D34245091

Pulled By: neerajprad

fbshipit-source-id: 1cd653c1d5c663346433e84fd0bbe2e590790908
(cherry picked from commit ef1da3b)
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 25, 2026
…ytorch#72993)

Summary:
Maintanance of pytorch#70377
Multiple modifications of the merged initial implementation of Wishart distribution.

Key modifications:
* torch/distributions/wishart.py: Clamp (Clip) float type values to calculate reciprocal in numerically stable manner, by using the eps value paired to each torch.dtype
* test/distributions/test_distributions.py: Test Wishart distribution implementation in numerically unstable zones, i.e df values are at ndim - 1 < df < ndim where ndim is the one dimenstion of the Wishart parameter & sample matrix.

Re-opened reverted PR pytorch#72059
cc neerajprad vadimkantorov

Pull Request resolved: pytorch#72993

Reviewed By: samdow

Differential Revision: D34853807

Pulled By: neerajprad

fbshipit-source-id: eb62dca19bf8a934fdf59b4ffc58587447fe8378
(cherry picked from commit 99240c0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla signed open source Reverted triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants