Skip to content

Fix Phi 4 tokenization#153

Merged
pcuenca merged 4 commits intomainfrom
phi-4-tokenizer
Jan 9, 2025
Merged

Fix Phi 4 tokenization#153
pcuenca merged 4 commits intomainfrom
phi-4-tokenizer

Conversation

@pcuenca
Copy link
Member

@pcuenca pcuenca commented Jan 9, 2025

No description provided.

@pcuenca pcuenca mentioned this pull request Jan 9, 2025
}
}

class PhiSimpleTests: XCTestCase {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test passes but these ones do not. I don't know if these tests were wrong all along or there's something else going on.

@pcuenca
Copy link
Member Author

pcuenca commented Jan 9, 2025

Tests pass, merging. Thanks a lot @DePasqualeOrg!

@pcuenca pcuenca merged commit a867fea into main Jan 9, 2025
1 check passed
@pcuenca pcuenca deleted the phi-4-tokenizer branch January 9, 2025 20:50
@DePasqualeOrg
Copy link
Contributor

Thank you @pcuenca!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants