Our existing documentation build process is ugly, hacky and brittle, and suffers from several problems:

• It's set up in such a way that changes on release branches (v0.34.x and v0.37.x at the moment) only get built when changes are pushed to main.
• Any changes to the build process and theme need to be backported across multiple branches, instead of being managed in a single place.

My recommendation is to split our docs build process out of the main repo into its own repo (https://github.com/cometbft/cometbft-docs), and there we would host:

• 1. Any code/config needed to build the docs (the static site generator config, theme, etc.).
• 2. Workflow(s) to build and push each version's documentation. What we ultimately want is:
  1. https://docs.cometbft.com to redirect the user to the latest stable release's documentation (must be configurable).
  2. We must be able to easily configure and build different versions' documentation to different subpaths at https://docs.cometbft.com, e.g.
     1. https://docs.cometbft.com/v0.34/ to host CometBFT v0.34's documentation from the v0.34.x branch.
     2. https://docs.cometbft.com/v0.37/ to host CometBFT v0.37's documentation from the v0.37.x branch.
     3. https://docs.cometbft.com/main/ to host CometBFT's documentation from the main branch.
• 3. The config that controls which versions should be built should ideally be located in a single file (i.e. single source of truth).
• 4. We need to build the documentation from the docs, spec and rpc/openapi folders for each release's documentation, e.g.:
  1. https://docs.cometbft.com/v0.34/ should show the contents of the docs folder
  2. https://docs.cometbft.com/v0.34/spec/ should show the contents of the spec folder
  3. https://docs.cometbft.com/v0.34/rpc/ should show the contents of the rpc/openapi folder
  4. All of these different docs must be available via the documentation navigation.
• 5. We need a version dropdown selector on the docs site that allows users to choose which version's documentation they want to read.
• 6. We need to be able to control which versions' documentation actually appears in that dropdown selector (i.e. we may want to expose the v0.34 and v0.37 docs via the version selector, but not the main docs).
• 7. We need to be able to search the documentation for each version separately (i.e. when searching while viewing the v0.34 docs, it should only return results for v0.34; when searching while viewing the v0.37 docs, it should only return results for v0.37; etc.)
• 8. There should be a way for us to trigger this rebuild in the docs repo every time we make changes to the docs in the main CometBFT repo. At the very least we could set up a cron job that executes this on a daily basis, as well as a manual workflow dispatch that would allow for manually triggering a docs site rebuild.

@aaronmw's kindly rethemed our docs site in #125, so I suspect that some of this would move across to the cometbft-docs repo if we go this route.