pip does this by default (you can opt out with --no-compile). Without this, you pay the cost on first import, which is undesirable in various situations. (Also a thing to make sure is apples to apples when benchmarking)