CYFRAGOVPL

/

Llama-PLLuM-70B-chat

like 2

Follow

Ministry of Digital Affairs of Poland 216

I represent a startup that provides inference services to our customers. We have been integrating PLLuM-70B for some of our clients and we have noticed poor quality of the chat_template you provide in tokenizer_config.json.

Here are our main issues with it:

1. The template forces user/assistant/user/assistant interleaved order of messages, raising exception if this validation fails.
2. The template does not include tool/function calling mechanisms.
3. Some tokens (eos_token, pad_token) are changed comparing to base model.

Since this PLLuM variant is based on Llama-70B, why change special tokens and impose such limitations? Especially blocking tool calling seems wrong, given how there are plans to use this model in mObywatel and other apps - wouldn't tool calling enable a lot of useful use cases?