@TheBloke please test and verify :)

@logicchains yea, having a separate file is not ideal. imo in an ideal wold we would have 1 convert.py and a subfolder with model specific code.

The delta between this file and convert-falcon-hf-to-gguf.py is just a few lines. I would really recommend consolidating into convert-falcon-hf-to-gguf (which might be a lot less work than consolidating into convert.py). Some of those changes would most likely make future falcon variations work more easily out of the box.

For example, for these lines

you can take a look at e276e4b to see how we've tried to consolidate this before:

Thanks for the updated script - it does work. Here is a sample run on M2 Ultra:

I agree with @akawrykow's suggestion to merge this into convert-falcon-hf-to-gguf.py instead of having a separate script.