At controller we set flag to throw exception only during first call of the action.

public class HomeController : Controller {
        private static int m_ThrowCounter;
        [ HttpGet ]
        [ Route("") ]
        public IActionResult CachedExceptionTest() {
            CachedExceptionTestModel model = new CachedExceptionTestModel();
            model.ThrowException = Interlocked.Increment(ref m_ThrowCounter) == 1;
            return View("Index", model);
        }
}

At view we throw exception when the flag set:

@model CachedExceptionTestModel
<cache expires-after="@TimeSpan.FromHours(1.0)">
    @if (Model.ThrowException) {
        throw new Exception();
    } else {
        @:OK
    }
</cache>

I suppose that CacheTagHelper should not cache exception occurs during first time call and just discard result. Second and following calls should not use cached exception, tag helper should execute inner contents and cache it only if it is not exception throwing.

In mine project for example this issue leads to the fact that when DB TimeoutException occurs during processing cached part of the page there no chance to see normal results in second later when DB load lowers. If exception occures in some page, all subsequent request to the page causes exception while cache tag not expire.

Test project attached:
CachedExceptionTest.zip