Copied from astropy/astropy#11595.

my thought for other packages is to essentially move the short script (if @adrn or others confirm it works) in astropy/astropy#11595 (comment) to sphinx_astropy and export numpydoc_xref_physical_type_aliases (perhaps renamed to include "astropy") into v1 so that in 3rd party packages' conf.py it gets imported. Then anyone who wants the physical-type xrefs can do numpydoc_xref_aliases.update(numpydoc_xref_physical_type_aliases). If they wanted different names, e.g. ['astropy-length'] instead of ['length'] if they support more than one unit package, then that can be done by
numpydoc_xref_aliases.update({"'astropy-"+k[1:]: v for k, v in numpydoc_xref_physical_type_aliases.items()}).
This would hide _units_and_physical_types and make xref plugin quite easy. I think implementing astropy/astropy#11595 (comment) directly in 3rd party packages should remain an experimental implementation and the next release of sphinx_astropy should codify the implementation for 3rd party packages.