zhouwg/kantv

1.I'm not AI expert and I'm learning on-device AI tech currently. all LLM models in this repo are created by official tools in llama.cpp(https://github.com/ggml-org/llama.cpp/blob/master/convert_hf_to_gguf.py, https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize). No fine-tuning or other specialized AI techniques were applied because I don't know how to use them at the moment.

2.LLM models in this repo are dedicated/validated for personal dev experimental.

3.LLM models in GGUF format which created by AI experts from ggml-org(https://huggingface.co/ggml-org) and lmstudio-community(https://huggingface.co/lmstudio-community) and unsloth(https://huggingface.co/unsloth) are strongly recommended.

4.about DeepSeek-R1-0528-Qwen3-8B-q4_k_m.gguf

• Model creator: deepseek-ai
• Original model: DeepSeek-R1-0528-Qwen3-8B
• steps to build DeepSeek-R1-0528-Qwen3-8B-q4_k_m.gguf

  export HF_ENDPOINT=https://hf-mirror.com (this is optional, might-be needed for developers in China)
  huggingface-cli download --resume-download deepseek-ai/DeepSeek-R1-0528-Qwen3-8B --local-dir DeepSeek-R1-0528-Qwen3-8B
  python convert_hf_to_gguf.py  DeepSeek-R1-0528-Qwen3-8B
  llama-quantize DeepSeek-R1-0528-Qwen3-8B-F16.gguf DeepSeek-R1-0528-Qwen3-8B-q4_k_m.gguf Q4_K_M

5.about MiMo-VL-7B-RL-q4_k_m.gguf

• Model creator: XiaomiMimo
• Original model: MiMo-VL-7B-RL
• steps to build MiMo-VL-7B-RL-q4_k_m.gguf(similar to DeepSeek-R1-0528-Qwen3-8B-q4_k_m.gguf)