Deprecated several functions at torch.nn.functional by weiyangfb · Pull Request #8748 · pytorch/pytorch

weiyangfb · 2018-06-21T16:48:51Z

summary

fixes Deprecate torch.nn.functional.tanh? #6245
deprecated tanh, sigmoid

torch/distributions/utils.py

torch/distributions/transforms.py

test/test_jit.py

soumith · 2018-06-21T17:32:54Z

what happens if user tries to do torch.nn.functional.sigmoid(x) now, is it an error or a deprecated warning? (it should be a deprecated warning)

weiyangfb · 2018-06-21T17:37:39Z

@soumith ah, I see. Now it is an error, I will make a deprecated warning

vishwakftw · 2018-06-21T18:12:22Z

Other functions in torch.nn.functional that could be deprecated:

selu
rrelu

Let me know what you think @soumith .

EDIT: there are not many changes if these functions are going to be deprecated, so can be a part of this PR too.

weiyangfb · 2018-06-25T19:48:50Z

@vishwakftw Yes I can deprecate selu and rrelu. But after that people can only use modules nn.SELU and nn.RReLU. Should we do this?

cc @soumith

soumith · 2018-06-25T19:55:23Z

wait, why would we deprecate relu and rrelu? we dont have torch.relu and torch.rrelu

weiyangfb · 2018-06-25T20:03:05Z

@soumith That's right. We will keep them for now then.

weiyangfb · 2018-06-25T20:03:16Z

@pytorchbot retest this please

vishwakftw · 2018-06-25T20:03:40Z

I thought we could deprecate torch.nn.functional.rrelu and torch.nn.functional.selu. Both are available as torch.rrelu and torch.selu (and their inplace versions) respectively.

This is how torch.nn.functional.selu works:

pytorch/torch/nn/functional.py

Lines 708 to 720 in 731273b

    
           def selu(input, inplace=False): 
        
               r"""selu(input, inplace=False) -> Tensor 
        
               Applies element-wise, 
        
               :math:`\text{SELU}(x) = scale * (\max(0,x) + \min(0, \alpha * (\exp(x) - 1)))`, 
        
               with :math:`\alpha=1.6732632423543772848170429916717` and 
        
               :math:`scale=1.0507009873554804934193349852946`. 
        
               See :class:`~torch.nn.SELU` for more details. 
        
               """ 
        
               if inplace: 
        
                   return torch.selu_(input) 
        
               return torch.selu(input)

weiyangfb · 2018-06-25T20:10:09Z

@vishwakftw Ah, I see! So they are just not documented yet, I will add them to the docs. This way I think functional.selu and functional.rrelu can be safely deprecated then.

input = torch.randn(2)
torch.selu(input)

tensor([1.4261, 0.2250])

input = torch.randn(2)
torch.rrelu(input)

tensor([-0.0398, -0.1174])

fmassa · 2018-06-26T16:11:12Z

So the goal now is to expose everything in the torch namespace?

weiyangfb · 2018-06-26T17:57:05Z

After offline discussion, we decide to move tanh and sigmoid from nn.functional to torch since they are more of common scientific functions, and keep the others at nn.functional for now.

weiyangfb · 2018-06-27T22:28:10Z

Failed Caffe2 CI tests look not related. Can I get a stamp on this?

weiyangfb · 2018-06-28T14:57:02Z

@pytorchbot retest this please

facebook-github-bot

@weiyangfb has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

vadimkantorov · 2018-06-30T19:17:47Z

relu also already is available on torch namespace: both torch.relu(torch.Tensor([1])) and torch.Tensor([1]).relu() work

same for softmax

soumith · 2018-07-01T13:49:47Z

@vadimkantorov we probably will remove that, please dont rely on undocumented features. the nn namespace is really the place for neural-net specifi functions

vadimkantorov · 2018-07-01T13:59:04Z

@soumith ok :) I was hesitating if it was a deficit of documentation or a blunder of auto bindings :)

Then it’s a consistency thing: softmax and sigmoid are used in often similar contexts

soumith · 2018-07-01T14:00:27Z

the latter :)

facebook-github-bot

@weiyangfb is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot

@weiyangfb has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Summary: 1. fixes pytorch#6245 2. deprecated tanh, sigmoid Closes pytorch#8748 Differential Revision: D8697975 Pulled By: weiyangfb fbshipit-source-id: f30714aa0611a1fe870040692f3dbcc8238aece9

weiyangfb requested review from apaszke, colesbury, ezyang, gchanan, soumith and zdevito as code owners June 21, 2018 16:48

soumith requested changes Jun 21, 2018

View reviewed changes

weiyangfb requested review from Yangqing, anderspapitto, bddppq, dzhulgakov, houseroad, jamesr66a and smessmer as code owners June 26, 2018 15:37

weiyangfb changed the title ~~Deprecated torch.nn.functional.tanh()~~ Deprecated several functions at torch.nn.functional Jun 26, 2018

weiyangfb force-pushed the deprecate_nn_functional_tanh branch from 8a4bf4f to 1d19ab6 Compare June 26, 2018 17:54

weiyangfb force-pushed the deprecate_nn_functional_tanh branch from 1d19ab6 to 4512e83 Compare June 26, 2018 18:20

soumith approved these changes Jun 29, 2018

View reviewed changes

facebook-github-bot reviewed Jun 29, 2018

View reviewed changes

facebook-github-bot reviewed Jul 2, 2018

View reviewed changes

weiyangfb added 6 commits July 2, 2018 12:58

removed tanh() from functional.py and rnn.py

6b51e03

using torch.tanh() instead

e1bc5f3

deprecated torch.nn.functional.sigmoid()

263ade8

use torch.sigmoid() instead

b007e95

added deprecated warnings

367445a

changed deprecated warnings

ce1d853

weiyangfb force-pushed the deprecate_nn_functional_tanh branch from 4512e83 to ce1d853 Compare July 2, 2018 19:59

facebook-github-bot reviewed Jul 2, 2018

View reviewed changes

facebook-github-bot closed this in cb1bfe9 Jul 2, 2018

weiyangfb deleted the deprecate_nn_functional_tanh branch July 2, 2018 23:09

Curt-Park mentioned this pull request Aug 29, 2019

Add sac MrSyee/pg-is-all-you-need#4

Merged

Conversation

weiyangfb commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

summary

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

soumith commented Jun 21, 2018

Uh oh!

weiyangfb commented Jun 21, 2018

Uh oh!

vishwakftw commented Jun 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weiyangfb commented Jun 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soumith commented Jun 25, 2018

Uh oh!

weiyangfb commented Jun 25, 2018

Uh oh!

weiyangfb commented Jun 25, 2018

Uh oh!

vishwakftw commented Jun 25, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

weiyangfb commented Jun 25, 2018

Uh oh!

fmassa commented Jun 26, 2018

Uh oh!

weiyangfb commented Jun 26, 2018

Uh oh!

weiyangfb commented Jun 27, 2018

Uh oh!

weiyangfb commented Jun 28, 2018

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

vadimkantorov commented Jun 30, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soumith commented Jul 1, 2018

Uh oh!

vadimkantorov commented Jul 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

soumith commented Jul 1, 2018

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

weiyangfb commented Jun 21, 2018 •

edited

Loading

vishwakftw commented Jun 21, 2018 •

edited

Loading

weiyangfb commented Jun 25, 2018 •

edited

Loading

vishwakftw commented Jun 25, 2018 •

edited

Loading

vadimkantorov commented Jun 30, 2018 •

edited

Loading

vadimkantorov commented Jul 1, 2018 •

edited

Loading