Help needed to run model on VLLm
#1
by
rbarria
- opened
Dear community,
I was trying to test this model on Colab T4,
and when I runned it:
import os
os.environ["VLLM_USE_V1"] = "0"
!vllm serve "Ti-Ma/TiMaGPT2-2022"
Got the error "PoptorchPipelinedGPT2LMHeadModel has no vLLM implementation and the Transformers implementation is not compatible with vLLM".
What can be done to run this model on vllm?
Thank you very much for your help.