There are some "FD leakage detected" errors in builds, and pypy appears to has a lot of them: https://travis-ci.org/pytest-dev/pytest/jobs/420632250#L658 I think CI should fail (at least in one job) in case this happens, i.e. on any warning.