ValueError: InternVLForConditionalGeneration has no vLLM implementation and the Transformers implementation is not compatible with vLLM.
#3
by
yang396409487
- opened
when I use vllm to infer, I got this error.
I find that the architectures of hf version is different from base version like [OpenGVLab/InternVL2_5-8B-MPO].
the base version architectures in config.json of is InternVLChatModel.
how to solve this?
model:OpenGVLab/InternVL2_5-8B-MPO-hf
vllm: 0.7.2
torch: 2.5.1
I also try below version, it does not work either
vllm: 0.8.5
torch: 2.6.0