fix(vllm): forward response_format to OpenAI-compatible API#4608
fix(vllm): forward response_format to OpenAI-compatible API#4608kartik-mem0 merged 1 commit intomem0ai:mainfrom
Conversation
The vllm provider accepted response_format as a parameter but never passed it into the request params, causing JSON parsing failures during memory extraction and update when the model returned free-form text instead of constrained JSON output. Closes mem0ai#4607 Made-with: Cursor
|
|
|
Thanks for the clean fix @rakheesingh — this looks good! The bug is clearly real, the fix follows the exact pattern used across all other providers, and the tests cover it well. Approving and merging this. One thing we noticed during review: |
I am gonna raise it right now |
The deepseek provider accepted response_format as a parameter but never passed it into the request params, causing JSON parsing failures during memory extraction and update when the model returned free-form text instead of constrained JSON output. This is the same bug that was fixed in vllm.py (mem0ai#4608), as noted by the maintainer during review. Closes mem0ai#4607 Made-with: Cursor
The azure_openai provider accepted response_format as a parameter but never passed it into the request params, causing JSON parsing failures during memory extraction and update when the model returned free-form text instead of constrained JSON output. This is the same bug class fixed in vllm.py (mem0ai#4608) and deepseek.py (mem0ai#4618), completing the fix across all OpenAI-compatible providers. Made-with: Cursor
Summary
vllmLLM provider acceptedresponse_formatas a parameter ingenerate_response()but never forwarded it into the actual API request paramsInvalid JSON response: Expecting value) during memory extraction/update because the model returned free-form text instead of constrained JSONresponse_formatto params, matching the pattern used byopenai.py,groq.py,deepseek.py, and all other OpenAI-compatible providersCloses #4607
Changes
mem0/llms/vllm.py— Added 2 lines to forwardresponse_formatinto request paramstests/llms/test_vllm.py— Added 2 new tests:test_generate_response_with_response_format— verifiesresponse_formatis passed to the APItest_generate_response_without_response_format— verifies it's omitted when not providedTest plan
openai.pyline 134-135