Chat Templates

A chat template is a Jinja2 snippet that formats messages into the string a model was trained on. For example:

>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-0.5B-Instruct")
>>> tokenizer.chat_template
"{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"
>>> tokenizer.apply_chat_template([{"role": "user", "content": "Hi!"}], tokenize=False)
'<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nHi!<|im_end|>\n'

In most cases you don’t need to worry about chat templates: models ship their template along with the tokenizer, and TRL applies it for you. The whole thing is transparent. But some TRL recipes rely on features that most shipped templates don’t include:

SFT with assistant_only_loss=True needs {% generation %} / {% endgeneration %} markers around assistant output, so the loss mask can target only assistant tokens.
GRPO with tool calls needs the template to be prefix-preserving: appending a tool message must not change how earlier messages are rendered.

TRL ships patched templates under trl/chat_templates/ for common families (Qwen, Llama, DeepSeek-V3, GPT-OSS, …) and swaps them in automatically for supported models. For any other model, you’ll need to patch its template yourself. The rest of this page catalogs what’s bundled.

Supported model families

TRL stores reference copies of the original templates so it can identify supported models at init and swap in a training template when needed. The following families are recognized: DeepSeek-V3, Gemma, GLM-4-MoE, GPT-OSS, Llama 3 / 3.1 / 3.2, Qwen2.5, Qwen3, Qwen3-VL, Qwen3.5.

Training templates

Patched templates that fix training-specific issues. Swapped in at init when tools are enabled (GRPO) or when assistant_only_loss=True (SFT).

deepseekv3_training.jinja

Patched DeepSeek-V3 template. Diff vs deepseekv3.jinja:

Uses | tojson on tool['function']['arguments'] so that arguments can be passed as a dict (the documented format per transformers docs). The original template uses raw string concatenation, which crashes on dict inputs.
Wraps assistant message output with {% generation %} / {% endgeneration %} markers for SFT assistant-only loss.

gemma_training.jinja

Patched Gemma template (shared by Gemma and Gemma2, which ship identical chat templates). Diff vs gemma.jinja:

Split the unified assistant output so that the <start_of_turn>model\n header (a prompt cue, not generated by the model) sits outside the generation block, and wrap the assistant content with {% generation %} / {% endgeneration %} markers for SFT assistant-only loss.

glm4moe_training.jinja

Patched GLM-4-MoE template. Diff vs glm4moe.jinja:

Require both <think> and </think> to be present before parsing, to avoid incorrect splitting when the model generates only one tag:

- {%- if '</think>' in content %}
+ {%- if '<think>' in content and '</think>' in content %}

Wrap assistant message output (including the thinking block and tool calls) with {% generation %} / {% endgeneration %} markers for SFT assistant-only loss.

qwen3_training.jinja

Patched Qwen3 template. Diff vs qwen3.jinja:

Require both <think> and </think> to be present before parsing, to avoid incorrect splitting when the model generates only one tag:

- {%- if '</think>' in content %}
+ {%- if '<think>' in content and '</think>' in content %}

Always include the thinking block regardless of message position. The original conditionally omits it based on loop.last, which changes the assistant rendering when a tool message is appended, breaking prefix-preservation:

- {%- if loop.index0 > ns.last_query_index %}
-     {%- if loop.last or (not loop.last and reasoning_content) %}
-         {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
-     {%- else %}
-         {{- '<|im_start|>' + message.role + '\n' + content }}
-     {%- endif %}
- {%- else %}
-     {{- '<|im_start|>' + message.role + '\n' + content }}
- {%- endif %}
+ {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}

Wrap assistant message output with {% generation %} / {% endgeneration %} so that return_assistant_tokens_mask=True produces correct masks for SFT assistant-only loss.

gptoss_training.jinja

Patched GPT-OSS template. Diff vs gptoss.jinja:

Wrap assistant message output with {% generation %} / {% endgeneration %} so that return_assistant_tokens_mask=True produces correct masks for SFT assistant-only loss.

llama3_training.jinja

Patched Llama 3 template. Diff vs llama3.jinja:

Wrap assistant message output with {% generation %} / {% endgeneration %} so that return_assistant_tokens_mask=True produces correct masks for SFT assistant-only loss.

qwen2_5_training.jinja

Patched Qwen2.5 template. Diff vs qwen2_5.jinja:

Wrap assistant message output with {% generation %} / {% endgeneration %} so that return_assistant_tokens_mask=True produces correct masks for SFT assistant-only loss.

Related utilities

See Chat Template Utilities for the helper functions (clone_chat_template(), is_chat_template_prefix_preserving, get_training_chat_template()) that operate on these templates.

Update on GitHub