Skip to content

Support for o200k_base and gpt-4o (omni) model#41

Closed
winzig wants to merge 5 commits intodmitry-brazhenko:mainfrom
winzig:main
Closed

Support for o200k_base and gpt-4o (omni) model#41
winzig wants to merge 5 commits intodmitry-brazhenko:mainfrom
winzig:main

Conversation

@winzig
Copy link
Contributor

@winzig winzig commented May 15, 2024

Notes:

  • I added in the new test plans for o200k based on output from tiktoken
  • I added a SetupO200k() in CompareBenchmark (and renamed Setup to SetupCL200k), but it's not flagged as the GlobalSetup, so not currently used/benchmarked. Wasn't sure if you wanted to re-benchmark with the new encoding?

winzig added 3 commits May 15, 2024 12:24
Accidentally removed a using statement in my last update
Still failing in a handful of tests of the new o200k
@winzig
Copy link
Contributor Author

winzig commented May 15, 2024

Fixed some issues that I found, all tests now passing.

@dmitry-brazhenko
Copy link
Owner

Thank you so much for contribution!

@dmitry-brazhenko
Copy link
Owner

dmitry-brazhenko commented May 17, 2024

I added minor fixes around failed tests in macos pipeline and will merge it here: #43

@dmitry-brazhenko
Copy link
Owner

Did it here: #43

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants