[Feature] Add /tokenize and /detokenize OpenAI-compatible endpoints#5711
[Feature] Add /tokenize and /detokenize OpenAI-compatible endpoints#5711adarshxs wants to merge 9 commits intosgl-project:mainfrom
Conversation
merrymercy
left a comment
There was a problem hiding this comment.
Can you add a test case?
|
Done @merrymercy |
|
Possible updates on this PR? Test failures don't seem to be related to this issue, and the added test cases are successful - i think it's due for a review. This feature would be really helpful. |
|
@zhyncs @merrymercy @zhaochenyang20 please review |
|
Is this PR still under review? It would be helpful to have it merged. |
|
Unfortunately there have been a few breaking changes to the OAI code so I've had to downgrade to 0.4.6-post5 to use this (and a tokenize endpoint is really useful!) |
|
Yeah the entire openai api interface has been refactored. I'll try implementing these endpoints for the new refractor ASAP and soon close this PR |
|
Thanks - your patch is the main reason I've been able to use sglang for my usecase! |
|
Thanks for your work, this will be really helpful! Is there an estimated timeline for merge or release? |
|
cc @thigger @Zyann7 @FeliceSchena @halaction apologies for the delay. opened a new PR with this feature compatible with the new openai interface. Hoping to have it merged soon hence i'll be closing this |
|
Thanks! I've been holding off upgrading sglang for this reason and will pull your new patch |
Motivation
requested in: #5653
Adds
/tokenizeand/detokenizeendpoints to the OpenAI-compatible API server.allows users to directly access the server's tokenizer for encoding text into token IDs and decoding token IDs back into text.
TokenizeRequest/ResponseandDetokenizeRequest/ResponsePydantic models.adapter.pyutilizing theTokenizerManager.http_server.py.add_special_tokens/skip_special_tokens./v1/path aliases.works like this:
Checklist