Had a [discussion](https://github.com/apache/datafusion/issues/18362#issuecomment-3523648340) with @nuno-faria on changing the cache accessor to support for non mut remove calls. Creating an issue for the same