common : fix XML tool calls with duplicate parameter keys#19753
common : fix XML tool calls with duplicate parameter keys#19753tomstokes wants to merge 1 commit intoggml-org:masterfrom
Conversation
Qwen3-Coder-Next can emit duplicate parameters in tool calls. The XML tool call parser stored parameters in an ordered_json object using operator[], which overwrites any previous values for the same key. This caused streaming diffs to fail when the duplicate key was partially streamed before the full duplicate was parsed. This change replaces operator[] with .emplace_back() in the XML tool call parser logic, ensuring that the streamed JSON object only grows monotonically. Duplicate keys are preserved and accurately streamed to the output. Signed-off-by: Tom Stokes <tomstokes@radixengineering.com>
|
Feels more like a grammar issue. It also doesn't appear relevant to the issue linked. |
|
Sorry, I linked the wrong issue. The correct issue is #19382 . I've updated the PR I agree that the LLM output shouldn't contain multiple parameters with the same key, but if it does then llama.cpp should produce valid output when translating it from XML to JSON. The current parser breaks when this situation occurs, which breaks tool calls. The (now correct) linked issue has several examples of people encountering the bug with Qwen3-Coder-Next. The output looks like this The linked issue #19382 has several other examples. The reports from that thread that disabling streaming fixed the issue were the clue that helped me find the problem. This PR would ensure correct streaming of XML tool calls and allow people to use Qwen3-Coder-Next with llama.cpp. It's a common complaint that Qwen3-Coder-Next and llama.cpp aren't working due to the intermittent failed tool calls from this problem. |
|
Give #19765 a try. It will also support duplicate keys, but they shouldn't appear if the grammar rules are correct. |
Qwen3-Coder-Next can emit duplicate parameters in tool calls. The XML tool call parser stored parameters in an ordered_json object using operator[], which overwrites any previous values for the same key. This caused streaming diffs to fail when the duplicate key was partially streamed before the full duplicate was parsed.
This change replaces operator[] with .emplace_back() in the XML tool call parser logic, ensuring that the streamed JSON object only grows monotonically. Duplicate keys are preserved and accurately streamed to the output.
Fixes #19382