Centering the design matrix cancels one of its singular values when n < p. This creates a numeric mess in the RidgeCV computation.
An example of the bad behaviour is given in:
https://gist.github.com/bthirion/5233416

I'm still not fully sure about the right solution. Comments welcome !
Best.