@ogrisel, @thomasjpfan, upon further inspection, I'm not that certain that the removal of params should go through the normal deprecation cycle (raise FutureWarning if used), since anybody using an extra keyword parameter in the fit method now receives an error.
For example, adding the parameter extra_param as follows:

from sklearn import svm
X = [[-2, -1], [-1, -1], [-1, -2], [1, 1], [1, 2], [2, 1]]
clf = svm.OneClassSVM()
clf.fit(X, extra_param='unused')

The following error is raised:

TypeError: fit() got an unexpected keyword argument 'extra_param'

To my knowledge, there is no extra keyword parameter that BaseLibSVM's fit method could accept. A couple of years ago the parameter class_weight was present with a deprecation message. This commit removed it, but param remained in OneClassSVM. Thus, the current status has been like this for a few years.

Please, let me know what do you think. Should we continue with the deprecation cycle or not?

I theory we should go through a deprecation cycle for this change. That means raising a warning to warn that the fit method will no longer accept these useless parameters in version 1.2.

Would we not consider this as a bug fix instead. For instance, a small typo in sample_weight could lead to something unexpected: estimator.fit(X, y, sample_weights=some_weights) while removing the kwargs would raise the right error.

After discussing with @ogrisel IRL, I am fine with the deprecation cycle. If a user makes a mistake in the spelling, he/she will at least get the warning that something is wrong.

Thanks for the suggestions @glemaitre 🤓