I'd really like to be able to install WordPress without any default themes or plugins.
I raised this here wp-cli/ideas#59 (raising a new issue on this repo as per the docs)

As @danielbachhuber mentioned --skip-content would be an appropriate flag.