Currently every layer that wants to cache itself or its children will recursively preroll their children and then evaluate if they can cache. The problem with this is that the children may also want to cache themselves and will do so on every frame even though the parent may already have cached them.

Consider an ImageFilterLayer (A) that has a child which is an ImageFilterLayer (B). If A decides to cache itself or its children then it does not need to cache B. However, it needs to visit B during Preroll in order for B to verify its state. As part of its Preroll, B may populate the layer cache with a copy of itself. Further complicating the logic here is that A may fail to cache itself for some reason that won't be known until B returns from the Preroll. That complication means that A can't do something to prevent B from Prerolling because it may be wrong.

The extra cache of B isn't necessarily bad for the frame where A initially does the caching as it will help during the rendering of A or its children. But, on subsequent frames where A is already cached, the cache of B is unneeded and could be freed to use less memory. This must be done carefully, though, so that we don't decide to re-cache B on the next frame.

It looks like the logic to prepare a layer cache will mark it as used even if it is never rendered, so we will avoid recreating the B cache by never releasing it. This may prevent the runaway "re-cache on every other frame" condition, but we still have the extra memory being used since we don't actually need the cache of B.