Without headers, the implementation uses the decodedAs helper https://github.com/haskell-servant/servant/blob/master/servant-client-core/src/Servant/Client/Core/HasClient.hs#L215-L230 which throws an exception if the server responds with an unexpected content type.

Whilst with headers, this extra check isn't done https://github.com/haskell-servant/servant/blob/master/servant-client-core/src/Servant/Client/Core/HasClient.hs#L244-L265

This seems like an easy fix to me.