@ejona86 @louiscryan and I had a conversation today. Here are the highlights:

Deprecate MutableHandlerRegistry, and later maybe place MutableHandlerRegistryImpl, which implements HandlerRegistry, in a util package.
ServerBuilder.addService() register handlers to an internal primary registry, which is immutable after built. The primary registry is not public API, so we are free to do optimizations for hpack. It doesn't support removeService() so we can flatten the lookup table (Consider removing ServerServiceDefinition, per the comment #1713) without a problem.
HandlerRegistry will stay as public API, but will only be used as a secondary dispatching mechanism which is only used when the primary handler registry doesn't find a match. This is the same idea as Stubby's default handler. HandlerRegistry can be a mutable lookup map (i.e., MutableHandlerRegistryImpl) or a totally dynamic one, which is up to the user. We will add a setFallbackHandlerRegistry(HandlerRegistry) to ServerBuilder.

This change will simplify our API surface while giving reasonable freedom of further optimization to us, and flexibility to the users.

Tracking Issue for Handler Registry being experimental. #933

Description