This PR was still pointing to develop, not master... hey look at that, it's no longer 200+ commits in a PR :D

I tested out this PR in my app, and for a 4k30hz video the display framerate goes from 2.5Hz up to 14Hz, quite an improvement.

Closing/Opening to trigger run in new CI

Okay, so considering #1466 is about to get merged, I dug in and figured out what other improvements are in this branch. The primary optimization here was pre-allocating the processing buffer, which is included as part of #1466. This branch does fix a bug (#667), improves docstrings, profiler and code clarity in a few places, and adds an Air-Speed-Velocity-based benchmark tool. These two branches have conflicting changes, so I made a new branch with those ironed out, and a few other things cleaned up. The hopeful list at the beginning of this PR is, as yet, undone, so I'm going to recommend we close this PR for now and make a new one out of the merged branches that better describes its functional intent.

This PR got super seeded by #1501 and #1466 thanks for the PR @campagnola !