Hugging Face's logo

kaitchup
/
OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit

Text Generation
Transformers
Safetensors
English
olmo2
auto-gptq
AutoRound
conversational
Inference Endpoints
4-bit precision
gptq
Model card Files Community

Model Details

This is allenai/OLMo-2-1124-7B-Instruct quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 4-bit. The model has been created, tested, and evaluated by The Kaitchup.

Details on the quantization process and how to use the model here: The Recipe for Extremely Accurate and Cheap Quantization of 70B+ LLMs

MMLU.png

Model Size.png

  • Developed by: The Kaitchup
  • Language(s) (NLP): English
  • License: Apache 2.0 license
Downloads last month
49
Safetensors
Model size
1.69B params
Tensor type
FP16
·
I32
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Collection including kaitchup/OLMo-2-1124-7B-Instruct-AutoRound-GPTQ-4bit