While investigating #579, @mulhod found weird inconsistencies when trying to manually run the code in that test in SKLL and when trying to run it in scikit-learn space with as little SKLL code as possible. Upon further investigation and using a more minimal example, I found that our decision to return 0 instead of NaN from our correlation functions is the issue here.

As this analysis shows, by returning 0s instead of NaNs, it is possible for SKLL to incorrectly choose the the least desirable hyperparameter configuration as the best option in some edge cases of this type where pearson may not be mathematically well-defined.

To fix this, we should just get rid of the line that converts NaNs to 0s. However, this might break some existing tests so they will need to be updated.