-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Open
Description
There was a breaking change to tokenizers::normalizers::utils::Sequence in v0.21.2. In v0.21.1, the struct had a get_normalizers method, which was removed in v0.21.2.
This breaks the build of mistral-rs v0.5.0, as cargo considers tokenizers v0.21.4 and v0.21.2 as semver-compatible to v0.21.1. The workaround is to manually downgrade the tokenizers dependency in the Cargo.lock using cargo update tokenizers --precise 0.21.1, but this will be undone by the next cargo update.
The proper fix would be to do a v0.21.5 release that brings those methods back.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels