Skip to content

fix(function_call): fallback to decode when batch decode options differ#15155

Merged
Fridge003 merged 5 commits intosgl-project:mainfrom
luqitao:fix_dp32_function_call
Dec 17, 2025
Merged

fix(function_call): fallback to decode when batch decode options differ#15155
Fridge003 merged 5 commits intosgl-project:mainfrom
luqitao:fix_dp32_function_call

Conversation

@luqitao
Copy link
Copy Markdown
Contributor

@luqitao luqitao commented Dec 15, 2025

DeepSeek v3.2 function calls require skip_special_tokens=False. Function call control markers (e.g. "| DSML |") are encoded as special tokens and must not be stripped during decoding.

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

DeepSeek v3.2 function calls require skip_special_tokens=False.
Function call control markers (e.g. "| DSML |") are encoded as special
tokens and must not be stripped during decoding.

Signed-off-by: luqitao <luqt@awcloud.com>
@luqitao luqitao force-pushed the fix_dp32_function_call branch from a60f472 to 1727419 Compare December 15, 2025 06:50
Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>
@JustinTong0323
Copy link
Copy Markdown
Collaborator

/tag-and-rerun-ci

@jimmy-evo
Copy link
Copy Markdown
Contributor

What is the performance impact?

@Fridge003 Fridge003 merged commit 0129c91 into sgl-project:main Dec 17, 2025
169 of 188 checks passed
@JustinTong0323
Copy link
Copy Markdown
Collaborator

What is the performance impact?

Slightly Slower but correct I think

@jimmy-evo
Copy link
Copy Markdown
Contributor

@JustinTong0323
I still get output like this, which missing |DSML|, even set disable_tokenizer_batch_decode=True

{
    "id": "c8adf5d758754176829c84a361c9761e",
    "object": "chat.completion",
    "created": 1766415495,
    "model": "deepseek/deepseek-v3.2",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "<function_calls>\n<invoke name=\"get_weather\">\n<parameter name=\"location\" string=\"true\">杭州</parameter>\n<parameter name=\"date\" string=\"true\">2025-12-05</parameter>\n</invoke>\n</function_calls>",
                "reasoning_content": "用户问的是后天,也就是12月5日的天气。我需要查询杭州和北京2025年12月5日的天气情况。现在时间是2025年12月3日下午2点30分,后天就是12月5日。我需要分别调用get_weather函数,获取杭州和北京12月5日的天气。现在开始查询。",
                "tool_calls": null
            },
            "logprobs": null,
            "finish_reason": "stop",
            "matched_stop": 1
        }
    ],
    "usage": {
        "prompt_tokens": 888,
        "total_tokens": 1025,
        "completion_tokens": 137,
        "prompt_tokens_details": {
            "cached_tokens": 832
        },
        "reasoning_tokens": 0
    },
    "metadata": {
        "weight_version": "default"
    }
}

Prozac614 pushed a commit to Prozac614/sglang that referenced this pull request Dec 23, 2025
jiaming1130 pushed a commit to zhuyijie88/sglang that referenced this pull request Dec 25, 2025
YChange01 pushed a commit to YChange01/sglang that referenced this pull request Jan 13, 2026
@luqitao luqitao deleted the fix_dp32_function_call branch February 11, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants