mlabonne

/

gemma-3-27b-it-abliterated

like 152

Would love to see a QAT version of Gemma 3 as well.

same, hope it makes it

please make the qat version

QAT-q4_0-gguf would be perfect for this

Please, sir :)

Hey good news, I abliterated all the QAT unquantized models:

https://huggingface.co/mlabonne/gemma-3-1b-it-qat-abliterated
https://huggingface.co/mlabonne/gemma-3-4b-it-qat-abliterated
https://huggingface.co/mlabonne/gemma-3-12b-it-qat-abliterated
https://huggingface.co/mlabonne/gemma-3-27b-it-qat-abliterated

They need to be quantized but the checkpoints are available.

Thanks for QATs.

I tested your newest 27b. Unfortunately v2 is lobotomized and broken compared to v1. when given the same prompt, it either:

gemma-3-27b-it-abliterated-v2.q4_k_m - doesn't follow instruction and quickly devolves into endless repetition loops.

gemma-3-27b-it-qat-abliterated.q5_k_m - follows instructions partially but less than v1, and with repetitive, broken grammar. also hallucinates context, goes off-topic.

v1 (gemma-3-27b-it-abliterated.q4_k_m) gives thorough, accurate and detailed replies in comparison.

Examples of bugged output:

Okay, here are some ways to explain the above topic. There are many aspects to consider and many different approaches to take.

Here' that topic broken down:

* **The Scope of the Problem** 
* **Here’s a breakdown of those aspects:**

Here's a breakdown of the topic:

* **The Basics**
* *Definition:* A term used in medicine to describe the process of development.
* Here's a breakdown of the above topics:

* **The Basics**
* **Here's a breakdown of the above topics as well as some more details:

* **The Basics**
* **Here's a breakdown of the above topics as well as some others:**
* And finally, here's a breakdown of the above topics:

* **The Basics**
* **Here's a breakdown of the above topics as well as some others:
* Here's a breakdown of the above topics:

These descriptions aim to capture the essence of war through language, while avoiding words like 'suffering', 'humanity' and 'compassion'. Here are some more examples:

      The devastation and destruction: War is often described in terms of its physical impact – the broken buildings, scorched earth, and loss of life.
      The chaos and disorder: War disrupts lives, economies, and societies.
      The cost and expense: War requires resources, manpower, and time.


Here are a few more ways to describe war, keeping in mind the original request:

      The outcome of war: The end result of war is often death and destruction.
      The effects of war: War has many consequences, including economic hardship and social upheaval.
      The purpose of war: Some believe that war is necessary for progress, while others see it as an inevitable part of human nature.

Thanks for flagging this @TheDevilishOne , it should be fixed now. It's something I couldn't see because the outputs I check are truncated. Hopefully, this new version behaves better.

Thanks @mlabonne , I'll test some more but my initial run of the newest 27B v2 Q4_K_M appears to still have bugs.

Would it be possible to apply your v1 abliteration method to the Gemma3 27B QAT? That or create a QAT of this v1 27B?

I found neither the 4B or 12B functioned well for me. But your Gemma3 27B v1 (non-QAT) works pretty much flawlessly. This is my favorite model currently.

Good work sir, you are paving new ground for sure.

Ah, interesting. Are you talking about the 4B and 12B v1 or v2?