I can't get the model to run.
Hi, I'm new in this world and I'm trying to run the model but I'm failing.
My code looks like:
import torch
from transformers import AutoModelForCausalLM
from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM
from deepseek_vl2.utils.io import load_pil_images
# specify the path to the model
model_path = "deepseek-ai/deepseek-vl2-tiny"
When I run it I get an error:
> python my_extraction.py
Traceback (most recent call last):
File "/Users/andresinaka/Desktop/LLMs/my_extraction.py", line 4, in <module>
from deepseek_vl2.models import DeepseekVLV2Processor, DeepseekVLV2ForCausalLM
ModuleNotFoundError: No module named 'deepseek_vl2'
Not sure what's going on, I'm following the readme...
I have torch installed:
pip show torch
Name: torch
Version: 2.5.1
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: [email protected]
License: BSD-3-Clause
Location: /Users/andres.canal/Desktop/LLMs/myenv/lib/python3.12/site-packages
Requires: filelock, fsspec, jinja2, networkx, setuptools, sympy, typing-extensions
Required-by: accelerate, easyocr, llms, torchvision
I have transformers installed:
pip show torch
Name: torch
Version: 2.5.1
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: [email protected]
License: BSD-3-Clause
Location: /Users/andres.canal/Desktop/LLMs/myenv/lib/python3.12/site-packages
Requires: filelock, fsspec, jinja2, networkx, setuptools, sympy, typing-extensions
Required-by: accelerate, easyocr, llms, torchvision
(myenv) andres.canal:LLMs[main] $ pip show transformers
Name: transformers
Version: 4.49.0.dev0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: [email protected]
License: Apache 2.0 License
Location: /Users/andres.canal/Desktop/LLMs/myenv/lib/python3.12/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by: llms
I also didn't manage to get it working, but seems I'm few seteps ahead of you. First of all, you should
git clone https://github.com/deepseek-ai/DeepSeek-VL2.git
then install module with following command according to ReadMe:
pip install -e .
After that you will notice it requires torch==2.0.1... and this torch version is available for Python 3.10 (not higher...) :)
Now I'm stuck at xformers module missing error:
File ~\python_virtual_envs\torch_test\lib\site-packages\deepseek_vl2\models\siglip_vit.py:16
14 from timm.models._manipulate import named_apply, checkpoint_seq, adapt_input_conv
15 from transformers.modeling_utils import is_flash_attn_2_available
---> 16 from xformers.ops import memory_efficient_attention
17 from functools import partial
20 if is_flash_attn_2_available():
ModuleNotFoundError: No module named 'xformers'
Thanks @beednarz-p100 !
I really can't understand why they don't make this more friendly to use with better documentation. I'm so used to Ollama that this feels like a nightmare hahahah...
I also didn't manage to get it working, but seems I'm few seteps ahead of you. First of all, you should
git clone https://github.com/deepseek-ai/DeepSeek-VL2.git
then install module with following command according to ReadMe:
pip install -e .After that you will notice it requires torch==2.0.1... and this torch version is available for Python 3.10 (not higher...) :)
Now I'm stuck at xformers module missing error:
File ~\python_virtual_envs\torch_test\lib\site-packages\deepseek_vl2\models\siglip_vit.py:16
14 from timm.models._manipulate import named_apply, checkpoint_seq, adapt_input_conv
15 from transformers.modeling_utils import is_flash_attn_2_available
---> 16 from xformers.ops import memory_efficient_attention
17 from functools import partial
20 if is_flash_attn_2_available():ModuleNotFoundError: No module named 'xformers'
Hi @beednarz-p100 , I was also stuck at xformers error. I tried a few solutions such as installing different xformers version based on your cuda version. You can get it from here https://github.com/facebookresearch/xformers#installing-xformers
My cuda version is 12.2 so tried to download xformers 12.4 but couldn't get to work.
Please let me know if you were able to get it to work
Hello
@miral-songhela
,
Deepseek-VL2 uses torch==2.0.1 so you should be able to make it work by downloading xformers==0.0.20 --> https://github.com/facebookresearch/xformers/issues/752#issuecomment-1555756372