[Bugfix] Gemma 4 chat template crash with missing tool name and tool id#42188
Conversation
|
Documentation preview: https://vllm--42188.org.readthedocs.build/en/42188/ |
There was a problem hiding this comment.
Code Review
This pull request updates the examples/tool_chat_template_gemma4.jinja template to use the boolean=True flag with the default filter when resolving tool names. This change ensures that the 'unknown' fallback is used for empty strings as well as undefined values. I have no feedback to provide as there were no review comments to evaluate.
|
cc @bbrowning Happy to add unit tests but this is really an edge case found while debugging #42189 |
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com>
|
@yzong-rh Do these changes need to make their way to the upstream Gemma4 official chat templates in the HF repo? If so, we'll need to coordinate with @lucianommartins alongside other upstream template changes being reviewed. |
I think so... Is there a channel we discuss and coordinate this? |
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com>
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com>
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com>
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com> Signed-off-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
…id (vllm-project#42188) Signed-off-by: Yifan <yzong@redhat.com>
Purpose
Gemma 4's chat template does not handle missing names in tool results correctly. The intended behavior is to default to "unknown" but it's kept as a
Nonevalue instead. As a result, later informat_tool_response_block, theNonetool name triggers a hard crash.This bug manifests on the legacy
tool_responsespath when tool name is missing. It manifests on theOpenAI Chat Completionspath when the following message does not contain anameand does not contain atool_call_id(or if thetool_call_iddoes not match the tool call).This defect is present in both
examples/tool_chat_template_gemma4.jinjaand default chat template from HF.Test Plan
repro.pyTest Result
crashes with
TypeError: can only concatenate str (not "NoneType") to strbefore this fix.Unit tests pass
Made with Cursor