See #2663 (comment).

This has been implemented before by @szotp at #2033 but wasn't merged. If we have to have something, this feels like a better option than the more destructive restart added in the last version.