By default CalibratedClassifierCV uses a LinearSVC as base_estimator. LinearSVC is non-deterministic, which makes testing annoying. I think the default value is not that important, and I'd vote to just fix the random_state in the LinearSVC() that is constructed.