This is most definitely my error that I forgot to add an @Configuration to my custom auto configuration.

The problem occurs when an auto configuration class (e.g. MyCustomAutoConfiguration) has a nested configuration class with @EnableConfigurationProperties is missing a @Configuration (the root class) then the Spring Boot application works correctly. However, when using the ApplicationContextRunner the behaviour is not the same.

See this repository for a reproducing example. In the example repo if you move the @EnableConfigurationProperties then tests that were failing before will pass.

If you think that there is nothing that Boot can do about this, feel free to close it. I was just confused why it was working in one instance, but not in the other