Lysandre

Great question @usbphone !

You're entirely correct that llama.cpp maintains its own implementation; we've been working hand-in-hand with the team to get better intercompatibility however: export between frameworks on both sides (from transformers to llama.cpp, and vice-versa).

Additionally, with the frequent VLM releases happening recently, we've been working directly with @ngxson to have the implementations day-0 (or week-0) in llama.cpp when the implementation already exists in transformers.

We won't have "automatic day-0 support" as the two libs have different backends; but we can still try by making the two more compatible, reducing the time needed to get there.