Hugging Face's logo

roleplaiapp
/
SmallThinker-3B-Preview-IQ3_M-GGUF

Text Generation
Transformers
GGUF
Safetensors
English
llama-cpp
imatrix
Q8_0
3b
SmallThinker
qwen
PowerInfer
code
math
chat
roleplay
nlp
Inference Endpoints
conversational
Model card Files Community

roleplaiapp/SmallThinker-3B-Preview-IQ3_M-GGUF

Repo: roleplaiapp/SmallThinker-3B-Preview-IQ3_M-GGUF
Original Model: SmallThinker-3B-Preview Organization: PowerInfer Quantized File: smallthinker-3b-preview-iq3_m-imat.gguf Quantization: GGUF Quantization Method: IQ3_M
Use Imatrix: True
Split Model: False

Overview

This is an imatrix GGUF IQ3_M quantized version of SmallThinker-3B-Preview.

Quantization By

I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. I hope the community finds these quantizations useful.

Andrew Webby @ RolePlai

Downloads last month
17
GGUF
Model size
3.4B params
Architecture
qwen2
View all files
Inference Examples
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for roleplaiapp/SmallThinker-3B-Preview-IQ3_M-GGUF

Base model

Qwen/Qwen2.5-3B
Finetuned
Qwen/Qwen2.5-3B-Instruct
Quantized
(99)
this model

Datasets used to train roleplaiapp/SmallThinker-3B-Preview-IQ3_M-GGUF