regression in silhouette score

Apparently I introduced a last minute regression here: https://github.com/scikit-learn/scikit-learn/commit/f0f174b74fd9684748e7c425b74909272e6ab22d#commitcomment-14775170

Not sure if that deserves a backport / bugfix release?
