Skip to content

Misc. bug: legal GBNF grammar leads to crash and makes server non-functional until restart #19845

@fiesh

Description

@fiesh

Name and Version

The following legal grammar works correctly:

thinkblock ::= <think> !</think>* </think>
any ::= .
root ::= (thinkblock | any)*

This slightly altered version that is syntactically valid and equivalent breaks llama:

thinkblock ::= <think> !</think>* </think>
any ::= .*
root ::= (thinkblock | any)*

For example:

podman run -it -v /tmp:/grammar:ro -v /llama:/llama:ro --entrypoint /app/llama-completion ghcr.io/ggml-org/llama.cpp:full -m /llama/qwen3/Qwen3VL-8B-Instruct-Q4_K_M.gguf --grammar-file /grammar/grammar.gbnf -p 'yolo'

starts babbling in Chinese with the former but does not output anything with the latter.

This is particularly bad with llama's server, as using the latter grammar will break the server completely until it's restarted.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

libllama (core library), Other (Please specify in the next section)

Command line

podman run -it -v /tmp:/grammar:ro -v /llama:/llama:ro --entrypoint /app/llama-completion ghcr.io/ggml-org/llama.cpp:full -m /llama/qwen3/Qwen3VL-8B-Instruct-Q4_K_M.gguf --grammar-file /grammar/grammar.gbnf -p 'yolo'

Problem description & steps to reproduce

Breaks server until restart

First Bad Commit

No response

Relevant log output

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions