Many probability distributions require sampling from the Gamma distribution, including: Gamma, Beta, and Dirichlet.

Since Gamma samplers have complex control flow (for rejection sampling) and are seldom a bottleneck in probabilistic algorithms, it should suffice to implement a CPU-only implementation at first. What is more important than a CUDA implementation is a reparameterized sampler so that stochastic gradients can be propagated through the sampler (see paper and reference implementation by @naesseth).

Tasks

• Implement torch.standard_gamma and distributions.Gamma #3841 CPU implementation of basic random_gamma(requires_grad=False)
• Implement reparameterized gradient for Gamma sampler #3978 Support reparameterized random_gamma(requires_grad=True)

Map of modifications

• aten/src/TH/THRandom.c/h random single numbers
• aten/src/TH/generic/THTensorRandom.c/h random tensors
• aten/src/ATen/Declarations.cwrap bindings for ATen
• torch/csrc/generic/methods/TensorRandom.cwrap bindings for torch.Tensor
• torch/autograd/variable.py - Variable
• torch/distributions.py - Distributions