[BUGFIX] Replace assert with ValueError for response_format validation in chat completions endpoint#35443
Conversation
Signed-off-by: Sergey Antonov <antonovsergey93@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Code Review
This pull request addresses a bug where an assert was used for validating the response_format for json_schema, which could lead to a 500 error. The change correctly replaces this with a ValueError, ensuring a proper 400 Bad Request is returned for invalid requests. The added test case verifies this behavior. I've added one suggestion to further improve the validation logic for json_schema to provide more specific error messages to the user.
Signed-off-by: Sergey Antonov <antonovsergey93@gmail.com>
…letions endpoint When the completions endpoint receives a request with response_format type 'json_schema' but without the required json_schema field, the server crashes with an AssertionError resulting in a 500 Internal Server Error. This is the same issue fixed for chat completions in vllm-project#35443, but for the /v1/completions endpoint. Replace assert statements with explicit ValueError raises so that the error is caught by create_error_response and returned as a proper 400 Bad Request. Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
…ion (vllm-project#34687) Signed-off-by: Andrii <askliar@nvidia.com> Co-authored-by: Andrii <askliar@nvidia.com>
Signed-off-by: Roi Koren <roik@nvidia.com>
vllm-project#35184) Signed-off-by: Daniel Salib <danielsalib@meta.com>
Signed-off-by: angelayi <yiangela7@gmail.com>
…3012) Signed-off-by: Chenyaaang <chenyangli@google.com>
…m-project#35400) Signed-off-by: NickLucche <nlucches@redhat.com>
…t#35413) Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: gnovack <gnovack@amazon.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
…lm-project#35424) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
…5369) Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
…n in completions endpoint (vllm-project#35456) Signed-off-by: umut-polat <52835619+umut-polat@users.noreply.github.com>
Signed-off-by: Max Hu <maxhu@nvidia.com> Signed-off-by: Max Hu <hyoung2991@gmail.com> Co-authored-by: Max Hu <maxhu@nvidia.com> Co-authored-by: Shang Wang <shangw@nvidia.com>
…parallelism (vllm-project#35410) Signed-off-by: jasonlizhengjian <jasonlizhengjian@gmail.com>
Signed-off-by: Sergey Antonov <antonovsergey93@gmail.com>
|
Documentation preview: https://vllm--35443.org.readthedocs.build/en/35443/ |
|
Closing PR in favor of #35514 |
Purpose
When the /v1/chat/completions endpoint receives a request with
response_formattypejson_schemabut without the required json_schema field, the server crashes with an AssertionError, resulting in a 500 Internal Server Error.Fixes #35438
This is the same class of issue addressed in #35456 for the
/v1/completionsendpointTest Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.