Metamask 9 adds a notification if the removed web3 API is used

This was added in #10039 if I understand correctly.

I would like to ask what is the condition that actually triggers this notification? Is it if I try to access window.web3 or window.web3.currentProvider? In our app, I do a check for the provider in this manner:

if (typeof ethereum !== 'undefined') {
    return ethereum
} else if (typeof web3 !== 'undefined') {
    return web3.currentProvider
} else {
   throw new Error(...)
}

If I remove mentions of window.web3, I still get that notification... is it just cached in the store or how can I be sure that the notification condition doesn't happen? Thanks!