Duplicate from Mungert/Phi-4-mini-instruct.gguf

Co-authored-by: Mungert <[email protected]>

.gitattributes

@@ -0,0 +1,40 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+phi-4-mini-q4_k_l.gguf filter=lfs diff=lfs merge=lfs -text
+phi-4-mini-bf16-q8.gguf filter=lfs diff=lfs merge=lfs -text
+phi-4-mini-bf16.gguf filter=lfs diff=lfs merge=lfs -text
+phi-4-mini-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+phi-4-mini-q8.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,31 @@
+---
+license: mit
+---
+
+# **Phi-4-mini-instruct GGUF Models**
+
+This repository contains the **Phi-4-mini-instruct** model quantized using a specialized branch of **llama.cpp**:  
+🔗 [ns3284/llama.cpp](https://github.com/ns3284/llama.cpp/tree/master)
+
+Special thanks to [@nisparks](https://github.com/nisparks) for adding support for **Phi-4-mini-instruct** in **llama.cpp**.  
+This branch is expected to be merged into the master branch soon, so once that happens, it's recommended to use the main **llama.cpp** repository instead.
+
+---
+
+## **Included Files**  
+
+### `phi-4-mini-bf16.gguf`
+- Model weights preserved in **BF16**.
+- Use this if you want to **requantize** the model into a different format.
+
+### `phi-4-mini-bf16-q8.gguf`
+- **Output & embeddings** remain in **BF16**.
+- All other layers quantized to **Q8_0**.
+
+### `phi-4-mini-q4_k_l.gguf`
+- **Output & embeddings** quantized to **Q8_0**.
+- All other layers quantized to **Q4_K**.
+- **Note:** No custom matrix quantization applied, so default **llama.cpp** quantization settings are used.
+
+### `phi-4-mini-q6_k.gguf`
+- All layers quantized to **Q6_K**, using **default quantization settings**.

phi-4-mini-bf16-q8.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5ac768c8e2f3853955ab6378de171f1c286a15d9b31ecccb598a580eaba70105
+size 4660795552

phi-4-mini-bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f9253d91a880739f770f987fa61ffc886c82ea82416f1323d1173d851046e43
+size 7680694432

phi-4-mini-q4_k_l.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:290e7a9d82926ff260ec537cd24f7223d630d9c5a3027a7d8068050b035d76c2
+size 2640722080

phi-4-mini-q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc50079a86c0a53de80c863c825c7903b618ade3840d6d387335c46c7c231953
+size 3155623072

phi-4-mini-q8.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdf04d4a2556e3271dc6e2d7187d5ebf7b4b61a42a4acb559930e3aecdd2cc41
+size 4084611232