[Tool Call][DSV32] Streamline function call parameters by Muqi1029 · Pull Request #14750 · sgl-project/sglang

Muqi1029 · 2025-12-09T17:24:06Z

Motivation

As titled and Issue #14711.

IMPORTANT: This is an initial version aimed at streamlining function call parameters for DeepSeek-V3.2. I have tested it, and it passes all tests in test.registered.function_call.test_function_call_parser.TestDeepSeekV32Detector. However, I added quite a few new logics, so I’m not fully confident that there aren’t other bugs. I’m sharing the code here to monitor and track any future bug reports.

Update on 2025.12.22:
Now this PR aims to fix bugs about #15278

Checklist

Format your code according to the Format code with pre-commit.
Add unit tests according to the Run and add unit tests.
Update documentation according to Write documentations.
Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
Follow the SGLang code style guidance.
Work with maintainers to merge your PR. See the PR Merge Process

gemini-code-assist · 2025-12-09T17:24:30Z

Summary of Changes

Hello @Muqi1029, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces significant enhancements to the DeepSeek-V3.2 function call detection and parsing mechanism, particularly for streaming scenarios. The core changes focus on enabling incremental parsing of function call parameters, supporting both XML and JSON formats, and refining the logic for detecting incomplete tool calls. These updates aim to provide a more robust and flexible system for handling tool calls as they are generated and streamed by the model.

Highlights

Incremental Parameter Parsing: Implemented a new method _parse_parameters_partially to handle both JSON and XML parameter formats incrementally, allowing for more robust streaming of function call arguments.
Streaming Tool Call Logic Refinement: The parse_streaming_increment method was significantly refactored to first emit the tool name and then stream parameter differences as they are detected, improving the real-time parsing of tool calls.
Enhanced Tool Call Detection: Modified the has_tool_call method and added invoke_end_token_prefixes to improve the detection of partial and complete DeepSeek-V3.2 tool calls.
Improved Test Coverage: Updated streaming tests to use a DeepSeek-V3.2 tokenizer for more accurate simulation of streaming input, and expanded test cases to cover more parameter types.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request refactors the DeepSeekV32Detector to enhance its streaming tool call parsing capabilities for DeepSeek V3.2 models. Key changes include introducing a new _parse_parameters_partially method to handle both JSON and XML parameter formats, and significantly updating the parse_streaming_increment logic to stream tool call arguments incrementally, sending the tool name first and then argument differences. The has_tool_call detection was also improved, and a new update_state method was added for managing parsing state. Corresponding test cases were updated to simulate streaming more accurately by generating chunks based on tokenizer output and expanding XML parameter test coverage. Review comments pointed out the need to correct a return type hint for _parse_parameters_partially, simplify the argument_diff calculation, move a transformers import to the top of the test file, and remove a commented-out line in the tests.

gemini-code-assist · 2025-12-09T17:26:16Z

+                    argument_diff = (
+                        func_args_raw[len(self._last_arguments) :]
+                        if func_args_raw.startswith(self._last_arguments)
+                        else func_args_raw
+                    )


This logic for calculating argument_diff can be simplified. Instead of checking startswith and then slicing, you can directly slice and the result will be an empty string if len(self._last_arguments) is greater than or equal to len(func_args_raw), which is a safe and more concise way to get the diff.

argument_diff = func_args_raw[len(self._last_arguments) :]

gemini-code-assist · 2025-12-09T17:26:16Z

            ),
        ]
        self.detector = DeepSeekV32Detector()
+        from transformers import AutoTokenizer


According to PEP 8, imports should be placed at the top of the file, not inside functions or methods. Moving from transformers import AutoTokenizer to the top of the file improves code readability and organization.

gemini-code-assist · 2025-12-09T17:26:16Z


        # Simulate streaming by splitting into small chunks
-        chunks = [text[i : i + 5] for i in range(0, len(text), 5)]
+        # chunks = [text[i : i + 5] for i in range(0, len(text), 5)]


This line of code is commented out and appears to be a remnant from a previous implementation. It should be removed to improve code cleanliness.

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

jxz542189 · 2025-12-10T02:37:27Z

Great! The test was successful. @Muqi1029

Muqi1029 · 2025-12-10T03:54:16Z

Great! The test was successful. @Muqi1029

Thanks for the feedback. The current version only supports streaming at the function-parameter level. Implementing full streaming would require introducing additional logic, which in my view would clutter the codebase (though there may be good tooling out there to help avoid messy conditional logic). Still, I believe this version can be a first step.

jxz542189 · 2025-12-10T05:14:28Z

OK, do you have any subsequent plans to achieve full streaming?

Muqi1029 · 2025-12-10T06:08:21Z

OK, do you have any subsequent plans to achieve full streaming?

Yes, I do have it. But I plan to wait until this version is validated and ready to merge first.

robscc · 2025-12-11T02:20:23Z

hi， I Have tested ur pr but found an issue

curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
  "model": "DeepSeek-V3.2",
  "messages": [
    {
      "role": "user",
      "content": "Calculate the derivative of the function 3x^2 + 2x - 1."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "calculate_derivative",
        "description": "Calculate the derivative of a polynomial function. Note that the provided function is in Python 3 syntax.",
        "parameters": {
          "type": "object",
          "properties": {
            "function": {
              "type": "string",
              "description": "The polynomial function."
            },
            "x_value": {
              "type": "number",
              "description": "The x-value at which the derivative is calculated. Optional, default to 0.00. This is a float type value.",
              "format": "float"
            }
          },
          "required": ["function"]
        }
      }
    }
  ],
  "stream":true,
  "temperature":1
}'
data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"I","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"'ll calculate the","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" derivative of the","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" polynomial function ","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"3x^","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2 + ","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2x","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" - 1","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" for you.\n\n","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"call_205edd924d6b4b4d867f1498","index":0,"type":"function","function":{"name":"calculate_derivative","arguments":""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"{"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419169,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"\n","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"f5dc2eea7603430b8b973380b378458b","object":"chat.completion.chunk","created":1765419170,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":"tool_calls","matched_stop":1}],"usage":null}

data: [DONE]

missing the arguments

Muqi1029 · 2025-12-11T02:47:34Z

arguments

Really thanks for your testing. Unfortunately, i cannot reproduce your error. I only changed the ip and port, the prompt is completely the same, my result is:

--header 'Content-Type: application/json' \
--data '{
  "model": "DeepSeek-V3.2",
  "messages": [
    {
      "role": "user",
      "content": "Calculate the derivative of the function 3x^2 + 2x - 1."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "calculate_derivative",
        "description": "Calculate the derivative of a polynomial function. Note that the provided function is in Python 3 syntax.",
        "parameters": {
          "type": "object",
          "properties": {
            "function": {
              "type": "string",
              "description": "The polynomial function."
            },
            "x_value": {
              "type": "number",
              "description": "The x-value at which the derivative is calculated. Optional, default to 0.00. This is a float type value.",
              "format": "float"
}'"temperature":1ed": ["function"]
data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null}
,"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"I","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"'ll","reasoning_content":null,"tool_calls":null},"lo
gprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" calculate","reasoning_content":null,"tool_calls":nu
ll},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" the","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" derivative","reasoning_content":null,"tool_calls":n
ull},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" of","reasoning_content":null,"tool_calls":null},"lo
gprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" the","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" function","reasoning_content":null,"tool_calls":nul
l},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"3","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"x","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"^","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" +","reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"x","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" -","reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"1","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" for","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" you","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":".\n\n","reasoning_content":null,"tool_calls":null},"
logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"ca
ll_e567ef4bf4c344aeb15a20b9","index":0,"type":"function","function":{"name":"calculate_derivative","arguments":""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"{"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"\"function\": \"3*x**2 + 2*x - 1\""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"}"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":"tool_calls","matched_stop":1}],"usage":null}

data: [DONE]

And I have requested this prompt several times(>=3), the result is consistently expected. Maybe you introduce other code?

jxz542189 · 2025-12-11T02:58:54Z

arguments

Really thanks for your testing. Unfortunately, i cannot reproduce your error. I only changed the ip and port, the prompt is completely the same, my result is:

--header 'Content-Type: application/json' \
--data '{
  "model": "DeepSeek-V3.2",
  "messages": [
    {
      "role": "user",
      "content": "Calculate the derivative of the function 3x^2 + 2x - 1."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "calculate_derivative",
        "description": "Calculate the derivative of a polynomial function. Note that the provided function is in Python 3 syntax.",
        "parameters": {
          "type": "object",
          "properties": {
            "function": {
              "type": "string",
              "description": "The polynomial function."
            },
            "x_value": {
              "type": "number",
              "description": "The x-value at which the derivative is calculated. Optional, default to 0.00. This is a float type value.",
              "format": "float"
}'"temperature":1ed": ["function"]
data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null}
,"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"I","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421217,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"'ll","reasoning_content":null,"tool_calls":null},"lo
gprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" calculate","reasoning_content":null,"tool_calls":nu
ll},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" the","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" derivative","reasoning_content":null,"tool_calls":n
ull},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" of","reasoning_content":null,"tool_calls":null},"lo
gprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" the","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" function","reasoning_content":null,"tool_calls":nul
l},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"3","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"x","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"^","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" +","reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"2","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"x","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" -","reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" ","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":"1","reasoning_content":null,"tool_calls":null},"logp
robs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" for","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":" you","reasoning_content":null,"tool_calls":null},"l
ogprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":".\n\n","reasoning_content":null,"tool_calls":null},"
logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"ca
ll_e567ef4bf4c344aeb15a20b9","index":0,"type":"function","function":{"name":"calculate_derivative","arguments":""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421218,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"{"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"\"function\": \"3*x**2 + 2*x - 1\""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":nul
l,"index":0,"type":"function","function":{"name":null,"arguments":"}"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"09b142b5c69f489fbb32fdc7db2fc972","object":"chat.completion.chunk","created":1765421219,"model":"DeepSeek-V3.2","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"log
probs":null,"finish_reason":"tool_calls","matched_stop":1}],"usage":null}

data: [DONE]

And I have requested this prompt several times(>=3), the result is consistently expected. Maybe you introduce other code?

My test results are the same as yours. Looking forward to full streaming.
#

Muqi1029 · 2025-12-11T03:01:59Z

@jxz542189 lol, thanks for the endorsement and for helping verify the PR’s correctness.

JustinTong0323 · 2025-12-14T01:33:06Z

/tag-and-rerun-ci

JustinTong0323 · 2025-12-14T02:16:33Z

I don't know why but I test with this req:

curl --location 'http://localhost:8080/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
  "model": "DeepSeek-V3.2",
  "messages": [
    {
      "role": "user",
      "content": "Calculate the derivative of the function 3x^2 + 2x - 1."
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "calculate_derivative",
        "description": "Calculate the derivative of a polynomial function. Note that the provided function is in Python 3 syntax.",
        "parameters": {
          "type": "object",
          "properties": {
            "function": {
              "type": "string",
              "description": "The polynomial function."
            },
            "x_value": {
              "type": "number",
              "description": "The x-value at which the derivative is calculated. Optional, default to 0.00. This is a float type value.",
              "format": "float"
            }
          },
          "required": ["function"]
        }
      }
    }
  ],
  "stream":true,
  "temperature":1
}'

And seems this cannot trigger tool on my side...

Muqi1029 · 2025-12-14T04:01:22Z

@JustinTong0323 I know why. One of your previous PR discard system prompt, making tools cannot be rendered in the prompt. This is an urgent bug, I have submitted a PR to fix it: #15092 .

And this bug is irrelated about this PR.

JustinTong0323 · 2025-12-16T03:08:35Z

@JustinTong0323 I know why. One of your previous PR discard system prompt, making tools cannot be rendered in the prompt. This is an urgent bug, I have submitted a PR to fix it: #15092 .

And this bug is irrelated about this PR.

My bad, my test script didn't catch that.... And I could not remember that..

JustinTong0323 · 2025-12-16T15:31:01Z

/rerun-failed-ci

JustinTong0323 · 2025-12-16T21:06:49Z

Hi @Muqi1029, I have implemented the parameter streaming version in #15278, and it also passed the test case you added in this PR. Would you kindly review it? If you find my PR acceptable, I will proceed to merge it and close this PR. Nevertheless, I would like to express my gratitude for your contribution!

Co-authored-by: Muqi Li <muqi1029@gmail.com> Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

Signed-off-by: Muqi Li <muqi1029@gmail.com>

Muqi1029 · 2025-12-22T07:03:26Z

@JustinTong0323 @Fridge003
Now I have used stricter test case and fixed bugs about #15278 .

You can checkout the test files and then run the test file, you will clearly see the bugs.

Can you review this PR again

JustinTong0323 · 2025-12-22T22:06:21Z

@JustinTong0323 @Fridge003 Now I have used stricter test case and fixed bugs about #15278 .

You can checkout the test files and then run the test file, you will clearly see the bugs.

Can you review this PR again

I appreciate your efforts and would like to apologize for the previous oversight. I would review this soon.

JustinTong0323 · 2025-12-22T22:06:31Z

/tag-and-rerun-ci

JustinTong0323 · 2025-12-23T21:36:01Z

/rerun-failed-ci

JustinTong0323 · 2025-12-24T00:05:17Z

Thank you truly for your contribution, I will be much more careful next time.

JustinTong0323 · 2025-12-25T14:37:17Z

/rerun-failed-ci

…4750) Signed-off-by: Muqi Li <muqi1029@gmail.com>

Muqi1029 added 9 commits December 7, 2025 21:42

initial commit

9464734

update

81eb615

update

fd2ec6e

update

3516aa2

update

bf1d920

update

f42f945

Merge branch 'main' into stream_func_args_dpskv32

08c5ed2

remove print log

d980477

fix

cfcd95c

Muqi1029 requested review from CatherineSue and JustinTong0323 as code owners December 9, 2025 17:24

github-actions Bot added the deepseek label Dec 9, 2025

gemini-code-assist Bot reviewed Dec 9, 2025

View reviewed changes

JustinTong0323 self-assigned this Dec 9, 2025

Apply suggestions from code review

4393497

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Merge branch 'main' into stream_func_args_dpskv32

55c3b3d

Muqi1029 added 2 commits December 10, 2025 23:35

Merge branch 'main' into stream_func_args_dpskv32

dd428e1

tiny fix

3715d41

jxz542189 mentioned this pull request Dec 13, 2025

[Roadmap] DeepSeek v3.2 (GLM 5) Optimization #15025

Open

40 tasks

Merge branch 'main' into stream_func_args_dpskv32

f4b816c

JustinTong0323 mentioned this pull request Dec 13, 2025

[Bug] deepseek v3.2 function call cannot output in a streaming manner #14711

Closed

5 tasks

Merge branch 'main' into stream_func_args_dpskv32

5737ca1

github-actions Bot added the run-ci label Dec 14, 2025

JustinTong0323 and others added 2 commits December 16, 2025 03:54

Merge branch 'main' into stream_func_args_dpskv32

5489363

fix trigger

74c8f35

JustinTong0323 added a commit to JustinTong0323/sglang that referenced this pull request Dec 16, 2025

pick tests from sgl-project#14750

fa45cca

Co-authored-by: Muqi Li <muqi1029@gmail.com> Signed-off-by: Xinyuan Tong <xinyuantong.cs@gmail.com>

Muqi1029 added 4 commits December 17, 2025 11:27

Merge branch 'main' into stream_func_args_dpskv32

93abae5

make tests more strict & fix minor bug

b0ed47e

Signed-off-by: Muqi Li <muqi1029@gmail.com>

Merge branch 'main' into stream_func_args_dpskv32

6a88f34

fix

786dc6d

Muqi1029 force-pushed the stream_func_args_dpskv32 branch from db40f83 to add63f5 Compare December 22, 2025 06:44

fix

349ae9e

Signed-off-by: Muqi Li <muqi1029@gmail.com>

Muqi1029 force-pushed the stream_func_args_dpskv32 branch from add63f5 to 349ae9e Compare December 22, 2025 06:49

Merge branch 'main' into stream_func_args_dpskv32

065932d

JustinTong0323 approved these changes Dec 24, 2025

View reviewed changes

Fridge003 merged commit 01bd0d3 into sgl-project:main Dec 26, 2025
147 of 154 checks passed

YChange01 pushed a commit to YChange01/sglang that referenced this pull request Jan 13, 2026

[Tool Call][DSV32] Streamline function call parameters (sgl-project#1…

475460a

…4750) Signed-off-by: Muqi Li <muqi1029@gmail.com>

Muqi1029 deleted the stream_func_args_dpskv32 branch April 5, 2026 03:42

Conversation

Muqi1029 commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Checklist

Uh oh!

gemini-code-assist Bot commented Dec 9, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist Bot Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Dec 9, 2025

Choose a reason for hiding this comment

Uh oh!

jxz542189 commented Dec 10, 2025

Uh oh!

Muqi1029 commented Dec 10, 2025

Uh oh!

jxz542189 commented Dec 10, 2025

Uh oh!

Muqi1029 commented Dec 10, 2025

Uh oh!

robscc commented Dec 11, 2025

Uh oh!

Muqi1029 commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jxz542189 commented Dec 11, 2025

Uh oh!

Muqi1029 commented Dec 11, 2025

Uh oh!

JustinTong0323 commented Dec 14, 2025

Uh oh!

JustinTong0323 commented Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Muqi1029 commented Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JustinTong0323 commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JustinTong0323 commented Dec 16, 2025

Uh oh!

JustinTong0323 commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Muqi1029 commented Dec 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JustinTong0323 commented Dec 22, 2025

Uh oh!

JustinTong0323 commented Dec 22, 2025

Uh oh!

JustinTong0323 commented Dec 23, 2025

Uh oh!

JustinTong0323 commented Dec 24, 2025

Uh oh!

JustinTong0323 commented Dec 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Muqi1029 commented Dec 9, 2025 •

edited

Loading

Muqi1029 commented Dec 11, 2025 •

edited

Loading

JustinTong0323 commented Dec 14, 2025 •

edited

Loading

Muqi1029 commented Dec 14, 2025 •

edited

Loading

JustinTong0323 commented Dec 16, 2025 •

edited

Loading

JustinTong0323 commented Dec 16, 2025 •

edited

Loading

Muqi1029 commented Dec 22, 2025 •

edited

Loading