-
-
Notifications
You must be signed in to change notification settings - Fork 10
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
It's going to let you install and run models using https://github.com/mlc-ai/mlc-llm
e.g.
from mlc_chat import ChatModule
from mlc_chat.callback import StreamToStdout
cm = ChatModule(model="Llama-2-7b-chat-hf-q4f16_1")
print(cm.generate(prompt="Three terrific names for a pet skunk"))
# Prints that out
print(cm.generate(prompt="Two more"))
# Prints out two moreReactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request