The one-liner also includes | |
handy support for generation prompts, but note that it doesn't add BOS or EOS tokens! | |
If your model expects those, they won't be added automatically by apply_chat_template - in other words, the | |
text will be tokenized with add_special_tokens=False. This is to avoid potential conflicts between the template and | |
the add_special_tokens logic. |