My GGUF-IQ-ARM-Imatrix quants for inflatebot/MN-12B-Mag-Mell-R1.

Mag Mell R1 was tested with Temp 1.25 and MinP 0.2. This was fairly stable up to 10K, but this might be too "hot". If issues with coherency occur, try increasing MinP or decreasing Temperature.

Use the ChatML prompt format.

Downloads last month: 700

GGUF

Model size

12.2B params

Architecture

llama

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

View +2 variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Base model

inflatebot/MN-12B-Mag-Mell-R1

Quantized

(21)

this model

Collection including Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Quantized Models (GGUF, IQ, Imatrix)

Collection

Various quantizations of models in the GGUF format. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 94 items • Updated Feb 19 • 57