Following discussion in #16743 I think it would be good to document and standardize how we skip tests that require network or download (this could also apply to tests that are skipped for other reason). Currently there are these mechanisms,

1. SKLEARN_SKIP_NETWORK_TESTS for some tests
2. network pytest marker for other tests, meaning that such tests can be deselected with -m "not network" pytest option
3. also the --skip-network pytest option, that skips tests with the network marker. The issue with this is that it requires a conftests.py, which is currently not available e.g. when scikit-learn is installed and tests are run with pytest --pyargs sklearn.

I would propose to remove 1, and standardize/document the use of either 2 or 3. If we go with 3 we might need to move conftests.py under the sklearn folder, although it's also not ideal.