@GaelVaroquaux @agramfort All right after several checks I noticed that all solver libraries already have their own default on max_iter and tol (I do have a doubt about the default of max_iter in liblinear though). Hence I suggest the following strategie :

• Default the max_iter and the tol to 'auto' in LinearRegression and LinearSVM
• Implement a set_params called somewhere along the way of the function cascade to the actual solver call. This set_param would exclude max_iter and tol from the parameters provided to the solver if tol and max_iter equal 'auto' but would integrate max_iter and tol if they are provided by the user and pass their value along for the solver to use it.
• Do some benchmark to check if the default behaviour of the solver generate convergence warnings
• Adapt some default if that's the case

What do you think ? -> just implement a function called in the .fit that maps solvers to tol and max_iter

On another subject I see that besides lbfgs, the tol for liblinear, newton-cg are also related to the value of the gradient of the function and not to value of the function. So while I can change that for lbfgs as scipy also expose an ftol, newton-cg and liblinear don't. There is probably some room for improvement there ?

What's the status on this?