roleplaiapp
/

SmallThinker-3B-Preview-IQ3_M-GGUF

Text Generation

Inference Endpoints

Model card Files Files and versions Community

roleplaiapp commited on 15 days ago

Commit

7c8e2f6

·

verified ·

1 Parent(s): d872e9a

update

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -35,12 +35,12 @@ tags:
 **Organization:** `PowerInfer`
 **Quantized File:** `smallthinker-3b-preview-iq3_m-imat.gguf`
 **Quantization:** `GGUF`
-**Quantization Method:** `Q8_0`
 **Use Imatrix:** `True`
 **Split Model:** `False`
 ## Overview
-This is an GGUF Q8_0 quantized version of [imatrix](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview).
 ## Quantization By
 I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.

 **Organization:** `PowerInfer`
 **Quantized File:** `smallthinker-3b-preview-iq3_m-imat.gguf`
 **Quantization:** `GGUF`
+**Quantization Method:** `IQ3_M`
 **Use Imatrix:** `True`
 **Split Model:** `False`
 ## Overview
+This is an imatrix GGUF IQ3_M quantized version of [SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview).
 ## Quantization By
 I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.