Skip to content

Misc. bug: Missing <think> tag in response (DeepSeek R1) #11861

@9chu

Description

@9chu

Name and Version

version: 4713 (a4f011e8)
built with x86_64-conda-linux-gnu-cc (Anaconda gcc) 11.2.0 for x86_64-conda-linux-gnu

I don't know whether it's a bug or not.

The latest Jinja chat template for the DeepSeek r1 model adds a <think>\n postfix to force the model into thinking.
However, this makes all the responses losing the heading <think> tag like this:

Image

I suggest manually adding the <think> prefix in response when add_generation_prompt = true.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

libllama (core library)

Command line

numactl --interleave=0-1 ./llama-server -ngl 0 --mlock --no-mmap --numa numactl -t 62 --port 10000 --host 0.0.0.0 -m ../../../DeepSeek-R1-UD-IQ2_XXS/DeepSeek-R1-UD-IQ2_XXS-00001-of-00004.gguf --jinja --chat-template-file ../../models/templates/llama-cpp-deepseek-r1.jinja --reasoning-format deepseek

Problem description & steps to reproduce

  1. Running llama-server
  2. Chatting with DeepSeek R1

First Bad Commit

No response

Relevant log output

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions