[c2][serialized test] Use tempfile during serialized test comparison#12021
Closed
ajyu wants to merge 1 commit intopytorch:masterfrom
Closed
[c2][serialized test] Use tempfile during serialized test comparison#12021ajyu wants to merge 1 commit intopytorch:masterfrom
ajyu wants to merge 1 commit intopytorch:masterfrom
Conversation
f3d68a9 to
8fbd8e5
Compare
8fbd8e5 to
29594ac
Compare
Contributor
Author
|
@houseroad for review |
29594ac to
75b6f12
Compare
houseroad
approved these changes
Sep 25, 2018
Member
|
Do we have any other file we should remove, too, something like you have done in this PR? |
Contributor
Author
|
I'll look through to double check for any other files we should remove. |
75b6f12 to
8f6ed21
Compare
Summary: Pull Request resolved: pytorch#12021 TestPilot runs stress tests in parallel. These fail for serialized tests because extracting (and subsequent deletion) of binary data during the process isn't threadsafe. Add a random suffix to extracted directory to make it much more reliable. We use numpy.random because random is seeded for TestPilot runs. Also remove some accidentally checked in zips of a test that we didn't end up including for now. Reviewed By: houseroad Differential Revision: D10013682 fbshipit-source-id: 842a02ff1bbe859dfdb0496858de1a09fd1573fd
8f6ed21 to
67f45f7
Compare
petrex
pushed a commit
to petrex/pytorch
that referenced
this pull request
Sep 26, 2018
* upstream/master: (117 commits) Add full namespace resolution in CAFFE_DURATION (pytorch#12065) T33898723: Simple put operators for caffe2 stats (pytorch#12057) add narrow() support for sparse tensors re: pytorch#8853 (pytorch#11342) Fix ONNX bug, add symbolic for full Enable tracing of tensor factories with an out argument Fix warnings emitted when testing distributions (pytorch#12038) Unify versions across setup.py, libtorch, and libcaffe2 (pytorch#12053) add autodiff expressions for common operations (pytorch#11832) Blob doesn't allow access to destroyCall anymore (pytorch#11548) IValue can store Blob (pytorch#11414) Move Blob to ATen/core (pytorch#11924) Use tempfile during serialized test comparison (pytorch#12021) fix segfault when grad to a hook fn is None (pytorch#12028) Fallback CreateMutex/AtomicIter operators for mkl-dnn Unify all *_EXPORT and *_IMPORT macros across c++ backend (pytorch#12019) Add safety asserts for methods on TensorImpl which don't work on Variable. (pytorch#12058) Make USE_IDEEP work again (pytorch#12026) Fix "identifier following the 'template' keyword does not refer to a template" (pytorch#12037) Delete some unused variables. (pytorch#12059) Support TypeIdentifier::name() (pytorch#12036) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
TestPilot runs stress tests in parallel. These fail for serialized tests because extracting (and subsequent deletion) of binary data during the process isn't threadsafe. Extract zips into tempfile to avoid this problem.
Also remove some accidentally checked in zips of a test that we didn't end up including for now
Differential Revision: D10013682