This issue was prompted by #6132. To explain the situation, let us consider the situation of convolution.

Today, we have a number of tests for convolution. The majority of these tests go through the "official" convolution interface, which is what all of our users use. However, inside the implementation of convolution, there are a large number of if-statements which specify which particular implementation we use, depending on some settings. This too, is also a good idea, because we want our users to get the best kernels without thinking about it.

However, when it comes time to test these kernels, we now have some very ugly code:

                # test that forwards of module runs correctly without cuDNN
                if self.cudnn:
                    with torch.backends.cudnn.flags(enabled=False):
                        module(input)
                        for p in module.parameters():
                            test_case.assertIsInstance(p, torch.cuda.FloatTensor)
                            test_case.assertEqual(p.get_device(), 0)

Essentially, we need to indirectly force the dispatch code one way or another, to make sure we test all of the actual kernel implementations. This makes it very easy to miss test configurations where you (a) forget to toggle between implementations or (b) the toggling code is wrong and you're not exercising a codepath that you actually wanted to test.

The basic situation is that there are two orthogonal things we want to test:

1. An integration test: we want to test that the actual function all of our users use for convolution actually works.
2. An implementation test: we want to test that each of our individual kernels is correct.

The current test suite conflates the two. It would be good if it did not.