The pandas_compat.py has over the years grown quite complex and a lot of pandas compatibility code, which probably can be simplified nowadays because of not supporting old pandas and Python versions anymore.

One aspect that I noticed while working on to_pandas was the complexity in the reconstruction of the .columns Index object of the resulting DataFrame. Right now that always goes through a MultiIndex (even for simple column names), which has quite some overhead of the simple case.