This is a large ticket that many people can collaborate on.

mlpack has very many tests, and many of these tests are probabilistic. This means that they can randomly fail, even when something isn't actually wrong with the code. The purpose of this ticket is to identify these tests, open issues to address those failing tests, and fix the tests. Typically this will involve understanding an algorithm, understanding the test, and adjusting the tolerance. This is a good way to get your hands dirty with mlpack, so I think this is a good starter ticket for anyone looking to get involved. I've labeled it all of 'easy', 'moderate', and 'difficult', because depending on the test, it might be easy to solve and it might not. Identifying failing tests should be an easy task too.

mlpack has a build server that runs a nightly matrix build. This can be found at

http://masterblaster.mlpack.org/job/mlpack%20-%20nightly%20matrix%20build/

If you click on any of the 'yellow' builds, you can get to the list of failing tests. You can also look at the Travis build history for mlpack, or run tests locally. We should identify tests that fail sometimes, and then open tickets to handle those individually. I will start with a couple here.

• RANNTest/RAModelTest: RAModel test fails for different types of trees. #734 (difficult)
• LARSTest/NoCholeskyTest: LARSTest/NoCholeskySingularityTest often fails for i386 #355 (difficult) -- only on i386
• GMMTest/GMMTrainEMOneGaussian
• VanillaNetworkTest in convolutional_network_test.cpp
• EmbeddedReberGrammarTest in recurrent_network_test.cpp
• LocalCoordinateCodingTest: LocalCoordinateCodingTestCodingStep hangs on sparc64 #943 (difficult) -- only on sparc64

Debugging tests that fail randomly can be done (or finding how often they fail) can be done by setting the random seed at the top of the test in question:

math::RandomSeed(std::time(NULL));

Then, you can run that individual test in a loop to see how often it fails:

while(true); do bin/mlpack_test -t TestSuiteName/TestCaseName; sleep 1; done

and this will run the test over and over again, and you can see how often it fails. That is usually a good first step to start understanding why the test is failing.

These failing tests are important to fix, because users (and new contributors) often get confused by failing Travis builds. We can close this bug when we've got the list down to 0. :)

If you'd like to contribute to this effort:

• Comment with the name of a test that seems to be commonly failing (and I'll update the list above).
• Open a separate issue for a test that is commonly failing (and I'll update the list above).
• Investigate a test that is commonly failing and submit a PR to fix it.