Skip to content

Improve reporting on pytest.#12610

Closed
ezyang wants to merge 2 commits intopytorch:masterfrom
ezyang:pr/improve-pytest-output
Closed

Improve reporting on pytest.#12610
ezyang wants to merge 2 commits intopytorch:masterfrom
ezyang:pr/improve-pytest-output

Conversation

@ezyang
Copy link
Copy Markdown
Contributor

@ezyang ezyang commented Oct 12, 2018

Before and after coming after I run the tests on CI

Signed-off-by: Edward Z. Yang ezyang@fb.com

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Signed-off-by: Edward Z. Yang <ezyang@fb.com>
@ezyang
Copy link
Copy Markdown
Contributor Author

ezyang commented Oct 12, 2018

Here's what it looks like after:

Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/predictor/predictor_test.py::TestPredictor::test_run PASSED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/predictor/predictor_test.py::TestPredictor::test_run_map PASSED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/test/blob_deallocation_test.py::BlobDeallocationTest::test PASSED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/test/do_op_test.py::DoOpTest::test_operator PASSED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/test/do_op_test.py::DoOpTest::test_reuse_workspace PASSED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_bvlc_alexnet <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_densenet121 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_inception_v1 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_inception_v2 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_relu_graph_big_batch <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_relu_graph_simple <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_resnet50 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_shufflenet <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_squeezenet <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_vgg16 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTOpTest::test_vgg19 <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/python/trt/test_trt.py::TensorRTTransformTest::test_resnet50_core <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [ 99%]
Oct 12 18:18:15 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_nccl_allgather PASSED [ 99%]
Oct 12 18:18:16 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_nccl_allreduce PASSED [ 99%]
Oct 12 18:18:16 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_nccl_broadcast PASSED [ 99%]
Oct 12 18:18:16 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_nccl_reduce PASSED [ 99%]
Oct 12 18:18:16 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_nccl_reduce_scatter PASSED [ 99%]
Oct 12 18:18:16 ../../../../usr/local/caffe2/lib/python2.7/dist-packages/caffe2/contrib/nccl/nccl_ops_test.py::NCCLOpsTest::test_timings <- ../../../../../../lib/python2.7/unittest/case.py SKIPPED [100%]
Oct 12 18:18:16 
Oct 12 18:18:16  generated xml file: /var/lib/jenkins/workspace/caffe2_tests/python/result.xml -
Oct 12 18:18:16 ---------- onnx coverage: ----------
Oct 12 18:18:16 Operators (passed/loaded/total): 81/81/118
Oct 12 18:18:16 ------------------------------------
Oct 12 18:18:16 Operator               Attributes
Oct 12 18:18:16                        (name: #values)
Oct 12 18:18:16 Asin                   No attributes
Oct 12 18:18:16 Slice                  axes: 3
Oct 12 18:18:16                        ends: 8
Oct 12 18:18:16                        starts: 7
Oct 12 18:18:16 Log                    No attributes
Oct 12 18:18:16 Sub                    No attributes
Oct 12 18:18:16 Min                    No attributes
Oct 12 18:18:16 Softsign               No attributes
Oct 12 18:18:16 Reshape                No attributes
Oct 12 18:18:16 Sum                    No attributes
Oct 12 18:18:16 Xor                    No attributes
Oct 12 18:18:16 Relu                   No attributes
Oct 12 18:18:16 Upsample               mode: 1
Oct 12 18:18:16                        scales: 1
Oct 12 18:18:16 Add                    axis: 2
Oct 12 18:18:16                        broadcast: 1
Oct 12 18:18:16 Abs                    No attributes
Oct 12 18:18:16 Pad                    mode: 3
Oct 12 18:18:16                        pads: 4
Oct 12 18:18:16                        value: 3
Oct 12 18:18:16 Split                  axis: 3
Oct 12 18:18:16                        split: 2
Oct 12 18:18:16 TopK                   k: 1
Oct 12 18:18:16                        axis: 0
Oct 12 18:18:16 Selu                   alpha: 1
Oct 12 18:18:16                        gamma: 1
Oct 12 18:18:16 Mul                    No attributes
Oct 12 18:18:16 LSTM                   hidden_size: 1
Oct 12 18:18:16                        activation_alpha: 0
Oct 12 18:18:16                        activation_beta: 0
Oct 12 18:18:16                        activations: 0
Oct 12 18:18:16                        clip: 0
Oct 12 18:18:16                        direction: 0
Oct 12 18:18:16                        input_forget: 0
Oct 12 18:18:16 ReduceMean             axes: 2
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 Acos                   No attributes
Oct 12 18:18:16 Gemm                   alpha: 2
Oct 12 18:18:16                        beta: 3
Oct 12 18:18:16                        broadcast: 1
Oct 12 18:18:16                        transA: 1
Oct 12 18:18:16                        transB: 1
Oct 12 18:18:16 Elu                    alpha: 1
Oct 12 18:18:16 Sqrt                   No attributes
Oct 12 18:18:16 Greater                No attributes
Oct 12 18:18:16 ArgMin                 axis: 1
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 Sin                    No attributes
Oct 12 18:18:16 BatchNormalization     epsilon: 4
Oct 12 18:18:16                        is_test: 1
Oct 12 18:18:16                        momentum: 3
Oct 12 18:18:16                        spatial: 0
Oct 12 18:18:16 Dropout                ratio: 3
Oct 12 18:18:16 Floor                  No attributes
Oct 12 18:18:16 Transpose              perm: 9
Oct 12 18:18:16 LeakyRelu              alpha: 3
Oct 12 18:18:16 Ceil                   No attributes
Oct 12 18:18:16 ReduceSum              axes: 2
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 Identity               No attributes
Oct 12 18:18:16 AveragePool            count_include_pad: 1
Oct 12 18:18:16                        kernel_shape: 8
Oct 12 18:18:16                        pads: 5
Oct 12 18:18:16                        strides: 6
Oct 12 18:18:16                        auto_pad: 0
Oct 12 18:18:16 Reciprocal             No attributes
Oct 12 18:18:16 Neg                    No attributes
Oct 12 18:18:16 Not                    No attributes
Oct 12 18:18:16 Squeeze                axes: 3
Oct 12 18:18:16 GlobalMaxPool          No attributes
Oct 12 18:18:16 Expand                 No attributes
Oct 12 18:18:16 Flatten                axis: 4
Oct 12 18:18:16 And                    No attributes
Oct 12 18:18:16 Cos                    No attributes
Oct 12 18:18:16 RNN                    activations: 2
Oct 12 18:18:16                        hidden_size: 1
Oct 12 18:18:16                        activation_alpha: 0
Oct 12 18:18:16                        activation_beta: 0
Oct 12 18:18:16                        clip: 0
Oct 12 18:18:16                        direction: 0
Oct 12 18:18:16 Atan                   No attributes
Oct 12 18:18:16 Mean                   No attributes
Oct 12 18:18:16 Tanh                   No attributes
Oct 12 18:18:16 ArgMax                 axis: 1
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 Gather                 axis: 2
Oct 12 18:18:16 LogSoftmax             axis: 5
Oct 12 18:18:16 Equal                  No attributes
Oct 12 18:18:16 Softplus               No attributes
Oct 12 18:18:16 Softmax                axis: 4
Oct 12 18:18:16 Exp                    No attributes
Oct 12 18:18:16 Div                    No attributes
Oct 12 18:18:16 GlobalAveragePool      No attributes
Oct 12 18:18:16 Or                     No attributes
Oct 12 18:18:16 PRelu                  No attributes
Oct 12 18:18:16 ConstantFill           input_as_shape: 1
Oct 12 18:18:16                        dtype: 0
Oct 12 18:18:16                        extra_shape: 0
Oct 12 18:18:16                        shape: 0
Oct 12 18:18:16                        value: 0
Oct 12 18:18:16 InstanceNormalization  epsilon: 2
Oct 12 18:18:16 Unsqueeze              axes: 3
Oct 12 18:18:16 Constant               value: 9
Oct 12 18:18:16 Clip                   max: 4
Oct 12 18:18:16                        min: 4
Oct 12 18:18:16 Conv                   dilations: 6
Oct 12 18:18:16                        group: 7
Oct 12 18:18:16                        kernel_shape: 11
Oct 12 18:18:16                        pads: 10
Oct 12 18:18:16                        strides: 7
Oct 12 18:18:16                        auto_pad: 0
Oct 12 18:18:16 Sigmoid                No attributes
Oct 12 18:18:16 Pow                    No attributes
Oct 12 18:18:16 ConvTranspose          dilations: 1
Oct 12 18:18:16                        group: 1
Oct 12 18:18:16                        kernel_shape: 1
Oct 12 18:18:16                        output_padding: 2
Oct 12 18:18:16                        pads: 1
Oct 12 18:18:16                        strides: 3
Oct 12 18:18:16                        auto_pad: 0
Oct 12 18:18:16                        output_shape: 0
Oct 12 18:18:16 MaxPool                kernel_shape: 7
Oct 12 18:18:16                        pads: 7
Oct 12 18:18:16                        strides: 6
Oct 12 18:18:16                        auto_pad: 0
Oct 12 18:18:16                        storage_order: 0
Oct 12 18:18:16 LRN                    alpha: 3
Oct 12 18:18:16                        beta: 2
Oct 12 18:18:16                        bias: 2
Oct 12 18:18:16                        size: 2
Oct 12 18:18:16 ReduceMin              axes: 1
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 MatMul                 No attributes
Oct 12 18:18:16 Less                   No attributes
Oct 12 18:18:16 Concat                 axis: 3
Oct 12 18:18:16 Max                    No attributes
Oct 12 18:18:16 Shape                  No attributes
Oct 12 18:18:16 Tan                    No attributes
Oct 12 18:18:16 ThresholdedRelu        alpha: 1
Oct 12 18:18:16 ReduceMax              axes: 1
Oct 12 18:18:16                        keepdims: 2
Oct 12 18:18:16 Size                   No attributes========== 1971 passed, 314 skipped, 3470 warnings in 1352.79 seconds ==========
Oct 12 18:18:18 + cd /usr/local/caffe2
Oct 12 18:18:18 + [[ -n '' ]]
Oct 12 18:18:18 ++ rm -f './crash/core.fatal_signal_as.*'
Oct 12 18:18:18 ++ rm -f './crash/core.logging_test.*'

Not perfect (there's still the onnx coverage report; see onnx/onnx#1506 ) but there are no more warnings. That's very nice.

@smessmer
Copy link
Copy Markdown
Contributor

Is blindly suppressing them the right thing to do? Are they meaningful?

@ezyang
Copy link
Copy Markdown
Contributor Author

ezyang commented Oct 16, 2018

Right now, most of the warnings are coming from onnx. In principle, it might be useful to squash these warnings, but this only makes sense if you're also wiling to Werror on them.

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ezyang is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@ezyang ezyang added the merged label Jun 25, 2019
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
Summary:
Before and after coming after I run the tests on CI

Signed-off-by: Edward Z. Yang <ezyang@fb.com>
Pull Request resolved: pytorch#12610

Differential Revision: D10419483

Pulled By: ezyang

fbshipit-source-id: 5543e971f8362e4cea64f332ba44a26c2145caea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants