-
Notifications
You must be signed in to change notification settings - Fork 873
Open
Labels
module: extensionIssues related to code under extension/Issues related to code under extension/module: user experienceIssues related to reducing friction for usersIssues related to reducing friction for userstriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
Hi,
I am trying to convert SmolLm-135M-Instruct model to .pte format and then run on an android device.
I have been successful in converting the model but executorch requires the tokenizer in either .bin format or .model format which can then be converted into .bin format. But on huggingface tokenizer.model or tokenizer.bin files are not present.
How would I go about converting the tokenizer.json file into the appropriate format.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
module: extensionIssues related to code under extension/Issues related to code under extension/module: user experienceIssues related to reducing friction for usersIssues related to reducing friction for userstriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Type
Projects
Status
To triage