Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,75 @@
+---
+license: other
+language:
+- en
+pipeline_tag: text-generation
+inference: false
+tags:
+- transformers
+- gguf
+- imatrix
+- DeepSeek-R1-Distill-Llama-8B
+---
+Quantizations of https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
+
+### Open source inference clients/UIs
+* [llama.cpp](https://github.com/ggerganov/llama.cpp)
+* [KoboldCPP](https://github.com/LostRuins/koboldcpp)
+* [ollama](https://github.com/ollama/ollama)
+* [text-generation-webui](https://github.com/oobabooga/text-generation-webui)
+* [jan](https://github.com/janhq/jan)
+* [GPT4All](https://github.com/nomic-ai/gpt4all)
+
+### Closed source inference clients/UIs
+* [LM Studio](https://lmstudio.ai/)
+* [Msty](https://msty.app/)
+* [Backyard AI](https://backyard.ai/)
+
+---
+
+# From original readme
+
+We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. 
+DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning.
+With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors.
+However, DeepSeek-R1-Zero encounters challenges such as endless repetition, poor readability, and language mixing. To address these issues and further enhance reasoning performance,
+we introduce DeepSeek-R1, which incorporates cold-start data before RL.
+DeepSeek-R1 achieves performance comparable to OpenAI-o1 across math, code, and reasoning tasks. 
+To support the research community, we have open-sourced DeepSeek-R1-Zero, DeepSeek-R1, and six dense models distilled from DeepSeek-R1 based on Llama and Qwen. DeepSeek-R1-Distill-Qwen-32B outperforms OpenAI-o1-mini across various benchmarks, achieving new state-of-the-art results for dense models.
+
+
+## How to Run Locally
+
+### DeepSeek-R1 Models
+
+Please visit [DeepSeek-V3](https://github.com/deepseek-ai/DeepSeek-V3) repo for more information about running DeepSeek-R1 locally.
+
+**NOTE: Hugging Face's Transformers has not been directly supported yet.**
+
+### DeepSeek-R1-Distill Models
+
+DeepSeek-R1-Distill models can be utilized in the same manner as Qwen or Llama models.
+
+For instance, you can easily start a service using [vLLM](https://github.com/vllm-project/vllm):
+
+```shell
+vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --tensor-parallel-size 2 --max-model-len 32768 --enforce-eager
+```
+
+You can also easily start a service using [SGLang](https://github.com/sgl-project/sglang)
+
+```bash
+python3 -m sglang.launch_server --model deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --trust-remote-code --tp 2
+```
+
+### Usage Recommendations
+
+**We recommend adhering to the following configurations when utilizing the DeepSeek-R1 series models, including benchmarking, to achieve the expected performance:**
+
+1. Set the temperature within the range of 0.5-0.7 (0.6 is recommended) to prevent endless repetitions or incoherent outputs.
+2. **Avoid adding a system prompt; all instructions should be contained within the user prompt.**
+3. For mathematical problems, it is advisable to include a directive in your prompt such as: "Please reason step by step, and put your final answer within \boxed{}."
+4. When evaluating model performance, it is recommended to conduct multiple tests and average the results.
+
+Additionally, we have observed that the DeepSeek-R1 series models tend to bypass thinking pattern (i.e., outputting "\<think\>\n\n\</think\>") when responding to certain queries, which can adversely affect the model's performance.
+**To ensure that the model engages in thorough reasoning, we recommend enforcing the model to initiate its response with "\<think\>\n" at the beginning of every output.**