jinja2 chat template is malformed

#13
by smcleod - opened

Your jinja2 chat template appears to be broken, when using it parsers fail with:

failed to parse chat template (defaulting to chatml): Expected comma in tuple at row 47, column 102:
{{ visible_text(m.content) }}
{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}}
                                                                                                     ^
{%- elif m.role == 'assistant' -%}

This will cause most inference servers to fall back to using a default template (e.g. ChatML) which will cause problems.

You should use the latest jinja2 package and use the latest versions of transformers and vLLM to support this template

smcleod changed discussion status to closed

Sign up or log in to comment