Add quantized models with per-model cards, MODELFILE, CLI examples, and auto-upload

.gitattributes

@@ -33,3 +33,12 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+Qwen3-Coder-30B-A3B-Instruct-f16_Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

MODELFILE

@@ -0,0 +1,25 @@
+# MODELFILE for Qwen3-Coder-30B-A3B-Instruct-GGUF
+# Used by LM Studio, OpenWebUI, GPT4All, etc.
+
+context_length: 32768
+embedding: false
+f16: cpu
+
+# Chat template using ChatML (used by Qwen)
+prompt_template: >-
+         <|im_start|>system
+       You are a helpful assistant.<|im_end|>
+         <|im_start|>user
+       {prompt}<|im_end|>
+         <|im_start|>assistant
+
+# Stop sequences help end generation cleanly
+stop: "<|im_end|>"
+stop: "<|im_start|>"
+
+# Default sampling
+temperature: 0.6
+top_p: 0.95
+top_k: 20
+min_p: 0.0
+repeat_penalty: 1.1

Qwen3-Coder-30B-A3B-Instruct-f16_Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58c5edfd3611ce69899a813bd5c6497c627ba419d687aacf2c269576ffdc0202
+size 11258611520

Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3830ae27570fa18f29973ded728f32ad55e6cf2dea29d91399fc01da5bf0b38
+size 14711848768

Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be281c99705609abcb058f1861c392cf169b241315bd3472c542a1dcd8c7ee0d
+size 13292470080

Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1077d608f7bb461e175591a8d4c6e3c36f1fb53d71d7e278df344ee4b41fcb0a
+size 18556688192

Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f6be7855b82c18d377d78e20ca269ca2571d9292f32560c9fc44edf4884d306
+size 17456011072

Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1bf15b812fde6f9770e00fd812c5da03ee722e0d0689096a912bdbfff4bc7a7
+size 21725583168

Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03696fd78795e19a20922ae01fa323d6c31f91455c04d6a9549f0b13ff36c474
+size 21080512320

Qwen3-Coder-30B-A3B-Instruct-f16_Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af2dd9723d4d7ccf5835df261f3a1502bcf845af602287ccef231dfdb122701e
+size 25092534080

Qwen3-Coder-30B-A3B-Instruct-f16_Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71e4443536ea01ac5bd0ca75c472969f70c17990e793e33dd57d31c520146a43
+size 32483934016

README.md

@@ -0,0 +1,64 @@
+---
+license: apache-2.0
+tags:
+  - gguf
+  - qwen
+  - llama.cpp
+  - quantized
+  - text-generation
+  - reasoning   - agent   - multilingual
+base_model: Qwen/Qwen3-Coder-30B-A3B-Instruct
+author: geoffmunn
+pipeline_tag: text-generation
+language:
+  - en
+  - zh
+  - es
+  - fr
+  - de
+  - ru
+  - ar
+  - ja
+  - ko
+  - hi
+---
+
+# Qwen3-Coder-30B-A3B-Instruct-GGUF
+
+This is a **GGUF-quantized version** of the **[Qwen/Qwen3-Coder-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct)** language model — a  Converted for use with \llama.cpp\, [LM Studio](https://lmstudio.ai), [OpenWebUI](https://openwebui.com), [GPT4All](https://gpt4all.io), and more.
+
+💡 **Key Features of Qwen3-Coder-30B-A3B-Instruct:**
+
+
+
+## Available Quantizations (from f16)
+
+| Level     | Quality       | Speed     | Size      | Recommendation |
+|----------|--------------|----------|-----------|----------------|
+| Q2_K | Minimal | ⚡ Fast | 19.5 GB | Only on severely memory-constrained systems. | | Q3_K_S | Low-Medium | ⚡ Fast | 22.2 GB | Minimal viability; avoid unless space-limited. | | Q3_K_M | Low-Medium | ⚡ Fast | 23.3 GB | Acceptable for basic interaction. | | Q4_K_S | Practical | ⚡ Fast | 27.0 GB | Good balance for mobile/embedded platforms. | | Q4_K_M | Practical | ⚡ Fast | 28.1 GB | Best overall choice for most users. | | Q5_K_S | Max Reasoning | 🐢 Medium | 31.5 GB | Slight quality gain; good for testing. | | Q5_K_M | Max Reasoning | 🐢 Medium | 32.2 GB | Best quality available. Recommended. | | Q6_K | Near-FP16 | 🐌 Slow | 36.5 GB | Diminishing returns. Only if RAM allows. | | Q8_0 | Lossless* | 🐌 Slow | 48.0 GB | Maximum fidelity. Ideal for archival. |
+> 💡 **Recommendations by Use Case**
+>
+  > - - 💻 **Standard Laptop (i5/M1 Mac)**: Q5_K_M (optimal quality)
+- 🧠 **Reasoning, Coding, Math**: Q5_K_M or Q6_K
+- 🔍 **RAG, Retrieval, Precision Tasks**: Q6_K or Q8_0
+- 🤖 **Agent & Tool Integration**: Q5_K_M
+- 🛠️ **Development & Testing**: Test from Q4_K_M up to Q8_0
+
+## Usage
+
+Load this model using:
+- [OpenWebUI](https://openwebui.com) – self-hosted AI interface with RAG & tools
+- [LM Studio](https://lmstudio.ai) – desktop app with GPU support
+- [GPT4All](https://gpt4all.io) – private, offline AI chatbot
+- Or directly via \llama.cpp\
+
+Each quantized model includes its own \README.md\ and shares a common \MODELFILE\.
+
+## Author
+
+👤 Geoff Munn (@geoffmunn)  
+🔗 [Hugging Face Profile](https://huggingface.co/geoffmunn)
+
+## Disclaimer
+
+This is a community conversion for local inference. Not affiliated with Alibaba Cloud or the Qwen team.

SHA256SUMS.txt

@@ -0,0 +1,9 @@
+58c5edfd3611ce69899a813bd5c6497c627ba419d687aacf2c269576ffdc0202  Qwen3-Coder-30B-A3B-Instruct-f16_Q2_K.gguf
+c3830ae27570fa18f29973ded728f32ad55e6cf2dea29d91399fc01da5bf0b38  Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_M.gguf
+be281c99705609abcb058f1861c392cf169b241315bd3472c542a1dcd8c7ee0d  Qwen3-Coder-30B-A3B-Instruct-f16_Q3_K_S.gguf
+1077d608f7bb461e175591a8d4c6e3c36f1fb53d71d7e278df344ee4b41fcb0a  Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_M.gguf
+3f6be7855b82c18d377d78e20ca269ca2571d9292f32560c9fc44edf4884d306  Qwen3-Coder-30B-A3B-Instruct-f16_Q4_K_S.gguf
+f1bf15b812fde6f9770e00fd812c5da03ee722e0d0689096a912bdbfff4bc7a7  Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_M.gguf
+03696fd78795e19a20922ae01fa323d6c31f91455c04d6a9549f0b13ff36c474  Qwen3-Coder-30B-A3B-Instruct-f16_Q5_K_S.gguf
+af2dd9723d4d7ccf5835df261f3a1502bcf845af602287ccef231dfdb122701e  Qwen3-Coder-30B-A3B-Instruct-f16_Q6_K.gguf
+71e4443536ea01ac5bd0ca75c472969f70c17990e793e33dd57d31c520146a43  Qwen3-Coder-30B-A3B-Instruct-f16_Q8_0.gguf