Just pushed some improvements. Enjoy!

• b3bdc62 main Make whisperfile 12% faster in GPU mode
• 0849f32 Get CUDA and Metal GPU working in whisperfile
• 94e9629 Add CLI program to whisperfile

OK the good news is that whisperfile works outstandingly well with the "large" model on cpu.

However I'm going to disable GPU support by default because it isn't working reliably. Here's CUDA.

Here's Apple Metal:

I'm not sure what's wrong. It's possible the problem will solve itself the next time I synchronize with llama.cpp upstream. I also checked and it definitely wasn't my performance optimization in the last change. The tiny whisper-tiny.en-q5_1.bin model does seem to work great on GPU (you can pass the --gpu auto flag to try it). So maybe the issue is related to how whisper-medium.en.bin and whisper-large-v3.bin are encoded. Any ideas?

Thank you @jart!! I must've missed porting some of the CUDA/Metal from whisperfile repo, appreciate you taking such quick care of it. Love the performance improvements on CPU as well!

It's possible could be something with the encoding, I've ran into some issues with the bin's before.. Also when I synced the whisper.cpp code I did not sanity check it (and I don't know if they are), so it may be an underlying issue? If you don't get to it first I'll take a closer look towards the end of the week/over the weekend