The right singular vectors of X can be computed by the eigenvalue decomposition of np.dot(X.T, X). So if n_features is not too large (say < 1000), this could be an efficient solver for TruncatedSVD.

Likewise, if n_samples << n_features, the left singular vectors could be obtained by the eigenvalue decomposition ofnp.dot(X, X.T) but then we would only be able to implement fit_transform, not transform.

See http://en.wikipedia.org/wiki/Singular_value_decomposition