run single-threaded gradgradcheck in test_nn#40999
Closed
ngimel wants to merge 1 commit intopytorch:masterfrom
Closed
run single-threaded gradgradcheck in test_nn#40999ngimel wants to merge 1 commit intopytorch:masterfrom
ngimel wants to merge 1 commit intopytorch:masterfrom
Conversation
albanD
approved these changes
Jul 6, 2020
Collaborator
albanD
left a comment
There was a problem hiding this comment.
LGTM.
Do we want to open a task to start running tests using multiprocessing if we run them in a single thread?
Collaborator
Author
|
Collaborator
|
Sounds good! Thanks for the details. |
Contributor
facebook-github-bot
left a comment
There was a problem hiding this comment.
@ngimel has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
Contributor
csarofeen
pushed a commit
to csarofeen/pytorch
that referenced
this pull request
Jul 7, 2020
Summary: Most time-consuming tests in test_nn (taking about half the time) were gradgradchecks on Conv3d. Reduce their sizes, and, most importantly, run gradgradcheck single-threaded, because that cuts the time of conv3d tests by an order of magnitude, and barely affects other tests. These changes bring test_nn time down from 1200 s to ~550 s on my machine. Pull Request resolved: pytorch#40999 Differential Revision: D22396896 Pulled By: ngimel fbshipit-source-id: 3b247caceb65d64be54499de1a55de377fdf9506
Collaborator
|
looks like this broke pytorch_linux_xenial_cuda11_0_cudnn8_py3_gcc7_test, will unland. |
malfet
reviewed
Jul 7, 2020
Comment on lines
-2049
to
+2051
| constructor=lambda: nn.Conv3d(4, 6, kernel_size=3, groups=2), | ||
| constructor=lambda: nn.Conv3d(2, 4, kernel_size=3, groups=2), | ||
| cpp_constructor_args='torch::nn::Conv3dOptions(4, 6, 3).groups(2)', | ||
| input_size=(2, 4, 4, 5, 4), | ||
| input_size=(1, 2, 3, 3, 3), |
Contributor
There was a problem hiding this comment.
@zasdfgbnm , looks like this change causes CuDNN v8 to fail with:
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.
Exception raised from operator() at /var/lib/jenkins/workspace/aten/src/ATen/native/cudnn/Conv.cpp:845
facebook-github-bot
pushed a commit
that referenced
this pull request
Jul 9, 2020
csarofeen
added a commit
to csarofeen/pytorch
that referenced
this pull request
Aug 16, 2020
This reverts commit 74f36b4.
2 tasks
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 24, 2026
Summary: Most time-consuming tests in test_nn (taking about half the time) were gradgradchecks on Conv3d. Reduce their sizes, and, most importantly, run gradgradcheck single-threaded, because that cuts the time of conv3d tests by an order of magnitude, and barely affects other tests. These changes bring test_nn time down from 1200 s to ~550 s on my machine. Pull Request resolved: pytorch#40999 Differential Revision: D22396896 Pulled By: ngimel fbshipit-source-id: 3b247caceb65d64be54499de1a55de377fdf9506
laurentdupin
pushed a commit
to laurentdupin/pytorch
that referenced
this pull request
Apr 24, 2026
Summary: Reland pytorch#40999 Pull Request resolved: pytorch#41147 Reviewed By: mruberry Differential Revision: D22450357 Pulled By: ngimel fbshipit-source-id: 02b6e020af5e6ef52542266bd9752b9cfbec4159
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Most time-consuming tests in test_nn (taking about half the time) were gradgradchecks on Conv3d. Reduce their sizes, and, most importantly, run gradgradcheck single-threaded, because that cuts the time of conv3d tests by an order of magnitude, and barely affects other tests.
These changes bring test_nn time down from 1200 s to ~550 s on my machine.