I noticed that while inspecting the conda package during my attempt to create a conda forge build.

The fix is likely as simple as adding benchmarks to packages=find_packages(exclude=[....]) in the setup.py file.