Skip to content

Eval bug: Parse errors increased with build 8515 (062cca58f) and current 8660 (d00685831) #21013

@chhil

Description

@chhil

Name and Version

./llama-cli --version
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
ggml_metal_library_init: loaded in 6.382 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name:   MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple9  (1009)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)
ggml_metal_device_init: simdgroup reduction   = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory    = true
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = false
ggml_metal_device_init: use residency sets    = true
ggml_metal_device_init: use shared buffers    = true
ggml_metal_device_init: recommendedMaxWorkingSetSize  = 55662.79 MB
version: 8515 (062cca58f)
built with AppleClang 17.0.0.17000604 for Darwin arm64

Built the current one now and same issue

version: 7960 (db6adb3c8)

Operating systems

Mac

GGML backends

Metal

Hardware

M4 Mac 64GB

Models

Qwen3-Coder-Next-Q4_K_M.gguf

Problem description & steps to reproduce

Random parse errors have increased (never used to get them from a version a week ago can't recall the exact version but).

e.g.
"Failed to parse input at pos 212: \n<tool_call>\n<function=grep>\n<parameter=pattern>\nclass.*MuxPool\n\n<parameter=path>\n/Volumes/SamsungT5/repositories/xyz/xyz-j/master\n\n\n</tool_call>"

"Failed to parse input at pos 22: <tool_call>\n<function=grep>\n<parameter=path>\n/Volumes/SamsungT5/repositories/xyz/xyz-j/master\n\n<parameter=pattern>\nmux\.visa\n\n<parameter=include>\n*.xml\n\n\n</tool_call>"

Opencode -> llama.swap --> llama.cpp

Opencode and llama.swap have been at the same version, llama.cpp was built and updated and started seeing this.

First Bad Commit

No response

Relevant log output

Logs
]

Notice the E failed to parse grammar in the log.

llama.cpp.log.txt

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions