A few estimators like Lasso support the warm_start option.

lasso = Lasso(warm_start=True)
scores = []
for alpha in alphas:
    lasso.set_params(alpha=alpha)
    lasso.fit(X_train, y_train)
    scores.append(lasso.score(X_test, y_test))

Due to how GridSearchCV is designed (use of clone, parallelism with n_jobs), setting the warm_start option wouldn't have any effect.

It would be nice to create a grid search object which can benefit from warm-start. n_jobs could still be supported but should be with respect to cross-validation folds.

This would require specifying with respect to what parameters the warm-starting must be done (e.g., alpha in the case of Lasso).