Description

The SAGA solver sag_solver fails to solve an edge case with L1 penalty taken from test_enet_toy. Depending on random_state, it does not converge to the correct solution. Instead, the solver gets stuck at zero, but thinks that it has converged after only one iteration. With a different random_state the correct result is obtained.

Steps/Code to Reproduce

import numpy as np
from sklearn.linear_model.sag import sag_solver

X = np.array([[-1.], [0.], [1.]])
Y = [-1, 0, 1]       # just a straight line

n_samples = X.shape[0]
alpha = 0.5
l1_ratio = 0.3
coef, n_iter, warm_start = \
    sag_solver(X, Y, loss='squared',
               alpha=n_samples*alpha*(1 - l1_ratio),
               beta=n_samples*alpha*l1_ratio,
               max_iter=1000, tol=1e-6, verbose=0, random_state=1,
               is_saga=True)
coef, n_iter

Expected Results

(array([0.50819], ...)

Actual Results

(array([0.]), 1)

Versions

System:
python: 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) [Clang 6.0 (clang-600.0.57)]
executable: .../python3.7
machine: Darwin-13.4.0-x86_64-i386-64bit

BLAS:
macros: NO_ATLAS_INFO=3, HAVE_CBLAS=None
lib_dirs:
cblas_libs: cblas

Python deps:
pip: 18.1
setuptools: 40.6.3
sklearn: 0.21.dev0
numpy: 1.15.4
scipy: 1.1.0
Cython: 0.29.2
pandas: 0.23.4