I think it would be nice if the 404 page on the blog had a link back to the homepage.

In the admin you can continue to navigate, but on the frontend there's no option to go anywhere.

If Ghost is up and serving 404s we can be reasonably certain the homepage works, so that seems like a good place to go next.