the docs seem to behave as expected and RTD built happily (and relatively quickly!)...is that a good sign?

Indeed! Just pushed up a commit to do the same thing for the Bag and DataFrame API docs too

This seems like a reasonable change to me. I don't see a lot of value in listing every method both on the page and in the toctree. @jacobtomlinson @jsignell thoughts?

My 2 cents is that from a UX perspective, I actually prefer to browse the page via the menu items to the right, and not the (now non-existent) menu items that used to be on the left. So from an end-user standpoint it works nicely for me

One small disadvantage is that when you are on the individual API function/method page, you no longer have the indication of the "active" navigation item in the sidebar.

When browsing to the overview page, the "API Reference" -> "DataFrame" is highlighted:

But if you then go to an individual page (eg https://dask--8238.org.readthedocs.build/en/8238/generated/dask.dataframe.DataFrame.html#dask.dataframe.DataFrame)

While on the current docs in master, you still have this (https://docs.dask.org/en/latest/generated/dask.dataframe.DataFrame.html):

I suppose this is to be expected (since for sphinx, this page is not located / linked to the "DataFrame API" page anymore). I am not fully sure how sphinx determines the "current" page indication, or whether this could be tweaked to still get this.

Yeah I think this is a good fix. The build time is down from 14.5 minutes to more like 10.5 minutes, so I think we should make this change.

Alright, I just pushed a commit which updates our docs to use https://github.com/executablebooks/sphinx-remove-toctrees. This worked well for me locally. Let's see how the RTD PR build goes