Concerning a segmentation fault I'm facing
Hello team, hope you're all well,
I am facing a segmentation fault when I run the model, I would like to debug it, what information do I need to provide you with to be able to help me walk through it?
I would like to provide you with enough information to make it a reproducible error, however I'm somewhat unfamiliar with cmake and builds and all that, so what information do I need to provide to be able to have someone help me debug through this issue?
Thank you in advance!
Are you running it via llama.cpp or ?
Yes I followed the instructions verbatim, built it using the fork, had to edit a couple of things in the cmakelists file to make it build, mainly for curl so nothing major
Built the binaries, then ran them with the following:
llama.cpp.qwen2vl/build/bin/llama-qwen2vl-cli -m weights/Qwen2.5-VL-3B-Instruct-iq2_xxs.gguf --mmproj weights/Qwen2.5-VL-3B-Instruct-mmproj-f16.gguf -p "prompt" --image id.png
All of this happened in a docker image so I think I can get it to be reproducible.
There is a slight error in the instructions it does not cd to the correct folder ie cd llama.cpp.qwen2vl before the git checkout. It will not switch to the correct branch with the current instructions .
it should be
Clone the latest llama.cpp Fork:
git clone https://github.com/HimariO/llama.cpp.qwen2vl.git
cd llama.cpp.qwen2vl
git checkout qwen25-vl
Try that and let me know if it works
I confirm there is still a seg fault even after building the correct checkout. It does this for all the lower quants but not for q4_0 .. Something has broken in the newer builds of https://github.com/HimariO/llama.cpp.qwen2.5vl ... Three choices 1: Use 4_0, other quants may work please let me know 2: checkout an older version of the HimariO (the lower quants where tested around 1 month ago) 3: wait for this repo to be fixed and merged into llama.cpp main.
Oh I see, alright thank you for taking the time to help, greatly appreciated, I will try the other fixes!
Possible fix try this version -> git checkout qwen25-vl-20250404 ... tested on Qwen2.5-VL-3B-Instruct-q3_k_s.gguf worked ok .
Still getting a segmentation fault with the following models, rebuilt everything using the branch you mentioned.
Qwen2.5-VL-3B-Instruct-f16-q4_k.gguf and Qwen2.5-VL-3B-Instruct-q4_0.gguf
I will try checking out an older version of the code as you mentioned.
Possible fix try this version -> git checkout qwen25-vl-20250404 ... tested on Qwen2.5-VL-3B-Instruct-q3_k_s.gguf worked ok .
I will try that thanks.
Make sure the checkout is switching correctly after git clone run git checkout qwen25-vl-20250404 . If it still not working try delete the folders and clone again. This checkout version is working for Qwen2.5-VL-3B-Instruct-iq2_xxs.gguf as well. The original git checkout latest qwen25-vl checkout seems to be broken.
Thanks for bringing this to my attention 👍
Will do, appreciate your time
Hello, I will be marking with as closed, I got it to work with this branch: qwen25-vl-20250404 on all of these models: Qwen2.5-VL-3B-Instruct-f16-q4_k.gguf and Qwen2.5-VL-3B-Instruct-q4_0.gguf and Qwen2.5-VL-3B-Instruct-iq2_xxs.gguf
Hello, I will be marking with as closed, I got it to work with this branch: qwen25-vl-20250404 on all of these models: Qwen2.5-VL-3B-Instruct-f16-q4_k.gguf and Qwen2.5-VL-3B-Instruct-q4_0.gguf and Qwen2.5-VL-3B-Instruct-iq2_xxs.gguf
Thank you for confirming that it works with branch