context: I've been working to minimize the total number of test dependencies in astropy. I arrived at the conclusion that the plugin provided by pytest-astropy couldn't be vendored in the core lib repo (via conftest.py) because these local configuration files are ignored when running the test suite in situ from an installed package (i.e., via pytest's --pyargs option). However, and as was previously discussed in #32, pytest-astropy, because it is also a meta package, brings in many other dependencies, some of which may be unused (see #72).
All things considered, I think it would be useful to split the package in two parts:

• a small, 0 dependencies (other than pytest) plugin whose only job would be to define @pytest.mark.(slow|hugemem), which could be used on its own
• a meta package whose sole job would be to bring a curated set of third-party dependencies, which we could continue advertising as a "batteries included" starter pack, but wouldn't have to dog-food it into astropy itself.

The meta package could very much include the lightweight one as a dependency. As a result, it would be backward compatible that pytest-astropy just becomes a pure meta package.
I think it would also neatly address #34: the purpose of each resulting package would be self-evident.