Presentlyu, SparsePCA used dict_learning or dict_learning_online on X.T, and sets its components_ to be the outputted code of these functions (which is sparse).

Though it is a simple way to reuse code from dict_learning.py, it has for drawback that it does not allow the user to learn in the sample direction, which prevents the use of partial_fit and is not adapted in a setting where data is streamed.

A solution for this would be to use another formulation for sparse PCA, found in [1] which enforces an elastic net constraint for D, and thus implies projecting D on the elastic net ball instead of the L2 ball as it is done today.

Efficient projection algorithms exists, as described in [1], [2].

[1] Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2010). Online learning for matrix factorization and sparse coding. The Journal of Machine Learning Research, 11, 19-60.
[2] Rodola, E., Torsello, A., Harada, T., Kuniyoshi, Y., & Cremers, D. (2013, December). Elastic net constraints for shape matching. In Computer Vision (ICCV), 2013 IEEE International Conference on (pp. 1169-1176). IEEE.