I've discovered this pytest plugin, which looks very interesting to me, but ideally I would want it to rerun tests a little more fine-grained, namely only when specific errors are detected, say HTTP response codes 5xx. I don't think this is already possible, right? If not maybe consider it as a feature request?