I'm submitting a feature request
I was fairly certain this was a thing, but it took me a long time to confirm to coworkers (and discover the nuances of using .babelrc vs. babel.config.js in the context babel-loader).

I think it's a good idea to document the fact that the config file is read automatically.