@agramfort @TomDLT @rth You might be interested.
Maybe, you can help with the final bits. For instance, there might be a problem with the stopping criterion of the cd solver. I had to decrease (making more strict) tol for sparse cases in comparison to the dense cases.

just to check did you observe any slowdown in the no sample weight case due to extract branching now present in the new code?

from sklearn.linear_model import ElasticNet
from sklearn.linear_model.tests.test_sparse_coordinate_descent import make_sparse_data

X, y = make_sparse_data(n_samples=1000, n_features=10_000)

%timeit ElasticNet().fit(X, y)

Results with %timeit -r50:

Note that I notices quite some variation of those timings.

@lorentzenchr beware that with such data and alpha=1 you get very sparse coef_ and very few iterations:

In [2]: %paste
from sklearn.linear_model import ElasticNet
from sklearn.linear_model.tests.test_sparse_coordinate_descent import make_sparse_data

X, y = make_sparse_data(n_samples=1000, n_features=10_000)
## -- End pasted text --

In [3]: clf = ElasticNet().fit(X, y)

In [4]: (clf.coef_ != 0).sum()
Out[4]: 2

In [5]: clf.n_iter_
Out[5]: 3

so computations like the lipschitz constants (norm(X, axis=0) ** 2) have a heavy weight in the total time.

In [19]:  %timeit ElasticNet(max_iter=0).fit(X, y)
84.3 ms ± 849 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

@mathurinm good point.
Next try:

from sklearn.linear_model import ElasticNet
from sklearn.linear_model.tests.test_sparse_coordinate_descent import make_sparse_data

X, y = make_sparse_data(n_samples=1000, n_features=10_000, n_informative=1000)

%timeit ElasticNet(alpha=0.01).fit(X, y)

Gives n_iter_=601.
Results:

perfect ! thx @lorentzenchr

just need another +1 to MRG this one. 🙏

issue #3702 was created in 2014. We are on shedule 😄
Thanks @lorentzenchr !

@lorentzenchr after thoughts I don't understand why in the sparse case R is scaled by sample_weights and X is not scaled, instead of both being scaled by sqrt(sample_weights) (here).

Is the current design adopted in order to have the same code to compute the gradient, with or without sample weights ?