stduhpf

/

google-gemma-3-27b-it-qat-q4_0-gguf-small

like 42

That's strange. I'll update a better updated version of this soon anyways, hopefully it will fix this issue too.

Yeah having the same issue

Actually I probably won't upload the new version, it's not performing as well as the current one in my benchmarks, surprisingly. I'll try figuring out the checksum issue tomorrow.

How much worse? 1-5% worse or 20%?

Like about 1% worse. But I was expecting it to be 1-2% better.

Ok I completed more through tests, and the new version seems to be actually on par with the current one, with better scores at some tests and worse at others. Results are always very much within the margin of error, so impossible to draw any definitive conlcusion on which one is "the best one". They're virtually equivalent. I won't bother uploading the new one unless it's the only way to fix this issue.

@blobbybob @dsent-zen What are the steps to reproduce?

@stduhpf well,

1. Download the file from here: https://huggingface.co/stduhpf/google-gemma-3-27b-it-qat-q4_0-gguf-small/blob/main/gemma-3-27b-it-q4_0_s.gguf — click "download."
2. Calculate sha256 checksum
3. Observe that it doesn't match the one listed on the download page (SHA256: f0c5f151111629511e7466a8eceacbe228a35a0c4052b1a03c1b449a8ecb39e8)

Another way:

1. Install LM Studio
2. Select the magnifying glass in the left panel ("discover")
3. Type Gemma 3 27b, find your model in the list
4. Click "download". After downloading and checking, you'll get checksum error.

If the sha256 hash was generated automatically at some point in the process, maybe a simple reupload will fix it. Right now it looks like the file was damaged either on your disk, or in transit. Maybe updating git and git-lfs is in order.

Ah, I see you've updated the repo. I'll recheck in an hour

Hmm the original file on my machine has the same sha256 as the one on the download page. (F0C5F151111629511E7466A8ECEACBE228A35A0C4052B1A03C1B449A8ECB39E8) So maybe some cosmic ray flipped a bit during the upload. It's very strange though.

Ah and of course, since the hashes technically match, I can't re-upload it. I guess it's a sign I should upload the new version instead. 😅

I got the same sha256 reported here: https://huggingface.co/stduhpf/google-gemma-3-27b-it-qat-q4_0-gguf-small/discussions/2

Ok I can reproduce. It's bizzare that Huggingface doesn't double check the checksums after upload...

10153600483328 bytes in the file there's a sequence of 16 bytes that got completely corrupted : 69 58 99 9b b8 ba 77 18 3b 1a 6c b0 54 ac 12 1d -> f1 04 e4 3c 00 00 40 00 f1 04 04 3d 00 00 40 00

I'm finally done uploading the updated version, let me know if it works now.

Looks fine on Ollama, breaks on load in LM Studio (but downloads correctly). Also, the vision doesn't seem to work, even though it downloads the mmproj in Ollama (sending an image gets an EOF error).

@Ixoth It works fine for me in LM studio, I just tried. Vison works too there. I think I heard people say the ollama implementation for vision models is different to the llama.cpp mmproj approach, so it's not cross compatible. Maybe there's a way to convert between the two, but I don't care enough about ollama to look too deep into it.