RichardErkhov/Devarui379_-_VersatiLlama-Llama-3.2-3B-Instruct-Abliterated-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

VersatiLlama-Llama-3.2-3B-Instruct-Abliterated - GGUF

• Model creator: https://huggingface.co/Devarui379/
• Original model: https://huggingface.co/Devarui379/VersatiLlama-Llama-3.2-3B-Instruct-Abliterated/

Name	Quant method	Size
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q2_K.gguf	Q2_K	1.27GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q3_K_S.gguf	Q3_K_S	1.44GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q3_K.gguf	Q3_K	1.57GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q3_K_M.gguf	Q3_K_M	1.57GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q3_K_L.gguf	Q3_K_L	1.69GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.IQ4_XS.gguf	IQ4_XS	1.71GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q4_0.gguf	Q4_0	1.79GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.IQ4_NL.gguf	IQ4_NL	1.79GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q4_K_S.gguf	Q4_K_S	1.8GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q4_K.gguf	Q4_K	1.88GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q4_K_M.gguf	Q4_K_M	1.88GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q4_1.gguf	Q4_1	1.95GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q5_0.gguf	Q5_0	2.11GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q5_K_S.gguf	Q5_K_S	2.11GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q5_K.gguf	Q5_K	2.16GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q5_K_M.gguf	Q5_K_M	2.16GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q5_1.gguf	Q5_1	2.28GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q6_K.gguf	Q6_K	2.46GB
VersatiLlama-Llama-3.2-3B-Instruct-Abliterated.Q8_0.gguf	Q8_0	3.19GB

Original model description:

base_model: - meta-llama/Llama-3.2-3B-Instruct license: cc-by-4.0 language: - en pipeline_tag: text-generation library_name: transformers

Model Card for Model ID

VersatiLlama-Llama-3.2-3B-Instruct-Abliterated

Model Description

Small but Smart

Fine-Tuned on Vast dataset of Conversations

Able to Generate Human like text with high performance within its size.

It is Very Versatile when compared for it's size and Parameters and offers capability almost as good as Llama 3.1 8B Instruct

Feel free to Check it out!!

Check the quantized model here: Devarui379/VersatiLlama-Llama-3.2-3B-Instruct-Abliterated-Imatrix-GGUF

[This model was trained for 5hrs on GPU T4 15gb vram]

• Developed by: Meta AI
• Fine-Tuned by: Devarui379
• Model type: Transformers
• Language(s) (NLP): English
• License: cc-by-4.0

Model Sources [optional]

base model:meta-llama/Llama-3.2-3B-Instruct

• Repository: Devarui379/VersatiLlama-Llama-3.2-3B-Instruct-Abliterated
• Demo: Use LM Studio with the Quantized version

Uses

Use desired System prompt when using in LM Studio The optimal chat template seems to be Jinja but feel free to test it out as you want!

Technical Specifications

Model Architecture and Objective

Llama 3.2

Hardware

NVIDIA TESLA T4 GPU 15GB VRAM