Add pos_weight argument to nn.BCEWithLogitsLoss (#5660)#6856
Merged
ssnl merged 2 commits intopytorch:masterfrom Jun 26, 2018
Merged
Add pos_weight argument to nn.BCEWithLogitsLoss (#5660)#6856ssnl merged 2 commits intopytorch:masterfrom
ssnl merged 2 commits intopytorch:masterfrom
Conversation
…tropy_with_logits (pytorch#5660) - Add an option to control precision/recall in imbalanced datasets - Add tests (but new_criterion_tests)
ezyang
reviewed
Apr 23, 2018
torch/nn/functional.py
Outdated
| target: Tensor of the same shape as input | ||
| weight (Tensor, optional): a manual rescaling weight | ||
| if provided it's repeated to match input tensor shape | ||
| pos_weight (Tensor, optional): a weight of positive examples. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
ezyang
reviewed
Apr 23, 2018
torch/nn/modules/loss.py
Outdated
| weight (Tensor, optional): a manual rescaling weight given to the loss | ||
| of each batch element. If given, has to be a Tensor of size | ||
| "nbatch". | ||
| pos_weight (Tensor, optional): a weight of positive examples. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
ezyang
approved these changes
Apr 23, 2018
Contributor
ezyang
left a comment
There was a problem hiding this comment.
Accept assuming the doc issues are fixed
`pos_weight` was moved to the end because it is the last argument in both `nn.BCEWithLogitsLoss` and `binary_cross_entropy_with_logits`
apaszke
reviewed
Apr 23, 2018
| is ``False``, returns a loss per input/target element instead and ignores | ||
| :attr:`size_average`. Default: ``True`` | ||
| pos_weight (Tensor, optional): a weight of positive examples. | ||
| Must be a vector with length equal to the number of classes. |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
apaszke
previously requested changes
Apr 23, 2018
| def __init__(self, weight=None, size_average=True, reduce=True, pos_weight=None): | ||
| super(BCEWithLogitsLoss, self).__init__(size_average, reduce) | ||
| self.register_buffer('weight', weight) | ||
| self.register_buffer('pos_weight', pos_weight) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Collaborator
|
@pytorchbot retest this please |
4 similar comments
Contributor
|
@pytorchbot retest this please |
Contributor
|
@pytorchbot retest this please |
Contributor
|
@pytorchbot retest this please |
Contributor
|
@pytorchbot retest this please |
petrex
pushed a commit
to ROCm/pytorch
that referenced
this pull request
Jun 26, 2018
* upstream/master: (42 commits) [c10d] No default device for ProcessGroupGloo (pytorch#8888) Fix default values for affine= in the docstrings of InstanceNormXd (pytorch#8895) Stop making dynamic allocations of PinnedMemoryAllocator. (pytorch#8896) [C++ API] Rework optimization package (pytorch#8815) Mention MPICH_MAX_THREAD_SAFETY=multiple. (pytorch#8580) Unify isViewable, handle n-dimensional empty tensors. (pytorch#8883) Add pos_weight argument to nn.BCEWithLogitsLoss (pytorch#5660) (pytorch#6856) [build] Enable clang-specific warnings only when using clang (pytorch#8869) Fix cmake cudnn autodetection (pytorch#8891) [c10d] Fix link order for building C++ tests (pytorch#8889) directly add_subdirectory(nanopb) from torch CMakeLists (pytorch#8870) [C++ API] Bag of fixes (pytorch#8843) [build] Raise in cmake when seeing NVCC{9/9.1} + GCC6 combo (pytorch#8863) Create avg_pool1d in ATen (pytorch#8880) throw error when grid_sample is passed unsupported mode (pytorch#8884) Allow autograd to work even when the shape of values cannot be determined (pytorch#8641) Make at::Tensor::to() const (pytorch#8839) [auto] Update onnx to 458c521 - Fix typo (onnx/onnx#1143) onnx/onnx@458c521 [Caffe2] Fix gradient_check on in-place ops (pytorch#8828) Fix as_strided_backward (pytorch#8721) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Multiplier
1 + (pos_weight - 1) * targetis the only significant difference.Notes:
F.binary_cross_entropy/nn.BCELoss. (It uses implementation from torch._C.) I can add a straightforward and numerically unstable implementation to this function but I'm not sure if it is really needed.pos_weightas last argument to prevent errors in the code that doesn't use names for keyword arguments. But it looks quite ugly.P.S. I proposed these changes in #5660