vcpkg install opencv shouldn't install opencv contrib in my opinion. It's quite separate from the main OpenCV distribution. It's a separate zip file download and installation. It takes significant amount of time to install as well and significant space on the HDD, for something many people that just want the core package will never need. contrib should definitely be a feature package, or, maybe even a separate package (like with boost who has modular packages).

The same goes for protobuf: opencv or opencv[core] shouldn't need protobuf - protobuf is only needed in very specific places in OpenCV, and I've used opencv for years without ever needing protobuf. It adds significant compile-time and install-space overhead.