Add import_models command to discover local MLX models.#6
Add import_models command to discover local MLX models.#6simonw merged 4 commits intosimonw:mainfrom
Conversation
llm_mlx.py
Outdated
|
|
||
| @mlx.command() | ||
| def import_models(): | ||
| cache_dir = Path(os.environ.get("HF_HOME", os.path.expanduser("~/.cache/huggingface"))) |
There was a problem hiding this comment.
I play with it when loading files from external disk.
|
I tested this like so: mv "$(uv run llm mlx models-file)" /tmp/llm-mlx.jsonuv run llm mlx models This confirmed I now have no models. Then with this PR: uv run llm mlx import-models
That sounds like too many! uv run llm models -q mlxOutputs: Some of those end with things like It's also picked up some non-LLM models like |
|
I wonder if there's a good way to detect which of those models are compatible with LLM?
|
|
I don't like hardcoding model_types in code, but I don't see other possibilities, what do you think? |
|
Another option could be to make this command interactive - so it shows you a list of models b ur asks you to confirm before adding them, maybe even lets you key up and key down and select toggle the ones you want. |
|
@simonw implemented, similar to huggingface-cli delete-cache. I excluded whisper and some VL models to reduce list. It seems ok now. WDYT? |
| if model_type in [ | ||
| "whisper", | ||
| "llava", | ||
| "paligemma", | ||
| "qwen2_vl", | ||
| "qwen2_5_vl", | ||
| "florence2", | ||
| "florence", | ||
| ]: | ||
| continue |
There was a problem hiding this comment.
Thanks! I had 100+ models, I had to cut that list down.
| window_size = os.get_terminal_size().lines - 5 | ||
|
|
||
| while True: | ||
| print("\033[H\033[J", end="") |
There was a problem hiding this comment.
Normally I'd suggest we use a cross-platform library like Rich here, but since this plugin only works on macOS there's no need for us to add another dependency.
Since ALL of those are |
I wonder if it's possible to detect that a model doesn't have chat templates setup and avoid suggesting it? |
|
Tests failed in Python 3.9: cat llm_mlx.py | llm -m o3-mini 'rewrite this file to not use case/match so it works in Python 3.9' Response: https://gist.github.com/simonw/16deb5b5cc66973ff04b2a2f3eaf00d9 |
|
I think we can land this. Things like "don't show options if everything is installed already" are purely nice-to-haves. |
But you can use tool to remove already imported models too. Maybe it was better calling it manage-models |
I bet it is, I will check this in the weekend |
Very basic import from local hugging face cache folder for all models containing mlx-community.
Closes #5