unsloth

/

DeepSeek-R1-GGUF

like 880

Follow

Unsloth AI 3.11k

What is the correct chat template for this model for Llama.cpp server? Do I need to use a jinja template?

• Thanks, Ash

It's in our model card and blog post! :) https://unsloth.ai/blog/deepseek-r1

Thanks for replying and creating the GGUF file. I did check the blog post earlier :-)

I could not figure how to convert the prompt format below to something that can be used by llama.cpp server app. I am using Open WebUI client with the llama.cpp server.
--prompt '<｜User｜>What is 1+1?<｜Assistant｜>' \

For mistral based models, I use --chat-template llama2 as parameter to it.

any update on this?

Found this out after days of toiling, only way i found to make it forcefully generate COT is to make my user prompts like this

\n (Prompt in here) \n\n

SPACES ARE IMPORTANT so the newline command doesnt get caught in the prompt. Hopefully it helps someone

EDIT: weird my thing is messed up but the prompt needs to look like this

< think >\n (Prompt in here) \n< /think >\n

remove spaces for before and after each think in between the ><

I use the default template, no custom server args