It seems to me that, the gradients compared in test_kernel_gradient are not equal.
K_gradient is the gradient with respect to the hyperparameter.
_approx_fprime calculates the gradient with respect to theta, which is log(hyperparameter)
I was running into issues while using _approx_fprime to test the correctness of my custom kernel, while setting the scale fixed it. But I am not sure how to add a non-regression test, or else I would have done it myself.