Skip to content

Raise error when each channel only has 1 value in batch norm#2961

Merged
soumith merged 2 commits intopytorch:masterfrom
ssnl:bn_1
Oct 3, 2017
Merged

Raise error when each channel only has 1 value in batch norm#2961
soumith merged 2 commits intopytorch:masterfrom
ssnl:bn_1

Conversation

@ssnl
Copy link
Copy Markdown
Collaborator

@ssnl ssnl commented Oct 3, 2017

Fixes #1381 by raising error when a channel observes only 1 value for a batch in batch norm.

@soumith soumith merged commit faa6fdf into pytorch:master Oct 3, 2017
@colesbury
Copy link
Copy Markdown
Member

This check isn't right. For example, we allow 1-element per channel in training mode is False.

@ssnl ssnl deleted the bn_1 branch October 4, 2017 17:21
@ssnl
Copy link
Copy Markdown
Collaborator Author

ssnl commented Oct 4, 2017

@colesbury good point, I'll write a PR to fix that case. Any other case this may be wrong?

ethanwee1 pushed a commit to ethanwee1/pytorch that referenced this pull request Feb 12, 2026
[REDUX] Refactor Apex build process to use the PyTorch JIT extension
flow ([pytorch#291](ROCm/apex#291))
([pytorch#296](ROCm/apex#296))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BatchNorm shouldn't use Bessel's correction for a batch size of 1

4 participants