Generates gibberish
I have tried the Q8 version which is on ollama and that works quite well. I'd say about 95% of the quality of the one hosted here: https://huggingface.co/spaces/Souvik3333/Nanonets-ocr-s
When I run the Q8 from Unsloth though, it's just gibberish; same for Q6. I've set the parameters correctly. Any idea why that would be? Do these quants work for anyone else?
Is this through Ollama?
Is this through Ollama?
Yes through Ollama with Msty . I tried the same models in LM Studio and they don't work properly for some reason. Not sure what Ollama does differently.
@danihend can you try redownloading and see if its fixed?
I downloaded in LM Studio as Ollama is not updated with the newest model yet. It works the same as others in LM Studio - not very well. I then imported it to Ollama and it just outputs nonsense like this in response to a image:
pdf
{
"bbox_2d": [1, 368, 497, 504],
"label": "ๅพ"
}
Not sure if it's due to an error on my part or what. Does it work for you with Ollama?