Transformers library does not recognize the llava_llama architecture

#6
by AllyBird - opened

I would like to try out this model for a financial application using the vllm approach, but I am encountering the following error:

Value error, The checkpoint you are trying to load has model type llava_llama but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

image.png

Here is the text form of the Trace:
Traceback (most recent call last):
File "/home/ec2-user/.local/bin/vllm", line 8, in
sys.exit(main())
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/cli/main.py", line 59, in main
args.dispatch_function(args)
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/cli/serve.py", line 58, in cmd
uvloop.run(run_server(args))
File "/home/ec2-user/.local/lib/python3.9/site-packages/uvloop/init.py", line 82, in run
return loop.run_until_complete(wrapper())
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/home/ec2-user/.local/lib/python3.9/site-packages/uvloop/init.py", line 61, in wrapper
return await main
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 1323, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/usr/lib64/python3.9/contextlib.py", line 181, in aenter
return await self.gen.anext()
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib64/python3.9/contextlib.py", line 181, in aenter
return await self.gen.anext()
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 177, in build_async_engine_client_from_engine_args
vllm_config = engine_args.create_engine_config(usage_context=usage_context)
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/engine/arg_utils.py", line 1018, in create_engine_config
model_config = self.create_model_config()
File "/home/ec2-user/.local/lib/python3.9/site-packages/vllm/engine/arg_utils.py", line 910, in create_model_config
return ModelConfig(
File "/home/ec2-user/.local/lib/python3.9/site-packages/pydantic/_internal/_dataclasses.py", line 123, in init
s.pydantic_validator.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
pydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig
Value error, The checkpoint you are trying to load has model type llava_llama but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date.

You can update Transformers with the command pip install --upgrade transformers. If this does not work, and the checkpoint is very new, then there may not be a release version that supports this model yet. In this case, you can get the most up-to-date code by installing Transformers from source with the command pip install git+https://github.com/huggingface/transformers.git [type=value_error, input_value=ArgsKwargs((), {'model': ..., 'model_impl': 'auto'}), input_type=ArgsKwargs]
For further information visit https://errors.pydantic.dev/2.11/v/value_error

Appreciate any advice! Thank you!

By the way, I already tried the indicated suggestion and Gemini's suggestion to use the most recent code from transformers.git.

getting same

Sign up or log in to comment