|
--- |
|
license: llama3.1 |
|
base_model: |
|
- deepseek-ai/DeepSeek-R1-Distill-Llama-8B |
|
- meta-llama/Llama-3.1-8B-Instruct |
|
--- |
|
|
|
The aim of this model is to retain the reasoning capabilities of <a href="https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B">DeepSeek-R1-Distill-Llama-8B</a>, while aligning more with the original <a href="https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct">Llama 3.1 model</a> on which it is based. |
|
|
|
As this model derives from Llama 3.1, the <a href="https://www.llama.com/llama3_1/license/">Llama 3.1 Community License Agreement</a> applies. |
|
|
|
Use the [DeepSeek Chat Prompt Template](https://docs.unsloth.ai/basics/tutorial-how-to-run-deepseek-r1-on-your-own-local-device#deepseek-chat-template) with this model. |
|
|
|
## 8B Safetensors BF16 format: |
|
Use with [transformers](https://huggingface.co/docs/transformers/en/index) as you would Llama 3.1, but use the [DeepSeek Chat Prompt Template](https://docs.unsloth.ai/basics/tutorial-how-to-run-deepseek-r1-on-your-own-local-device#deepseek-chat-template) as you would with the original DeepSeek-R1-Distill-Llama models. |
|
|
|
Use model id ___BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b___ |
|
|
|
[Or download files from here](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/tree/main) |
|
|
|
## 8B GGUF Quantised versions: |
|
|
|
Use these with [Llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/) or [Kobold.cpp](https://github.com/LostRuins/koboldcpp). |
|
Thanks to [mradermacher](https://huggingface.co/mradermacher) for converting these from the [safetensors](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/tree/main) format. |
|
|
|
| Filename | Type | Size | Quality | |
|
| -------- | ---------- | --------- | ----------- | |
|
| [LlamaAligned-DeepSeekR1-Distill-8b-Q4_K_M.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-8b.Q4_K_M.gguf?download=true) | Q4_K_M | 4.92GB | OK quality, default. | |
|
| [LlamaAligned-DeepSeekR1-Distill-8b-Q8_0.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-8b.Q8_0.gguf?download=true) | Q8_0 | 8.54GB | Best quality quantised version. | |
|
| [LlamaAligned-DeepSeekR1-Distill-8b-Q6_K.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-8b.Q6_K.gguf?download=true) | Q6_K | 6.6GB | High quality. | |
|
| [LlamaAligned-DeepSeekR1-Distill-8b-Q5_K_M>.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-8b.Q5_K_M.gguf?download=true) | Q5_K_M> | 5.73GB | Good quality. | |
|
| [LlamaAligned-DeepSeekR1-Distill-8b-Q3_K_S.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-8b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-8b.Q3_K_S.gguf?download=true) | Q3_K_S | 3.66GB | Lower quality. | |
|
|
|
|
|
## 70B Safetensors BF16 format: |
|
Use with [transformers](https://huggingface.co/docs/transformers/en/index) as you would Llama 3.3, but use the [DeepSeek Chat Prompt Template](https://docs.unsloth.ai/basics/tutorial-how-to-run-deepseek-r1-on-your-own-local-device#deepseek-chat-template) as you would with the original DeepSeek-R1-Distill-Llama models. |
|
|
|
[Or download files from here](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/tree/main) |
|
|
|
## 70B GGUF Quantised versions: |
|
|
|
Use these with [Llama.cpp](https://github.com/ggerganov/llama.cpp), [LM Studio](https://lmstudio.ai/) or [Kobold.cpp](https://github.com/LostRuins/koboldcpp). |
|
Thanks to [mradermacher](https://huggingface.co/mradermacher) for converting these from the [safetensors](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/tree/main) format. |
|
|
|
| Filename | Type | Size | Quality | |
|
| -------- | ---------- | --------- | ----------- | |
|
| [LlamaAligned-DeepSeekR1-Distill-70b-Q4_K_M.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-70b.Q4_K_M.gguf?download=true) | Q4_K_M | 42.5GB | OK quality, default. | |
|
| LlamaAligned-DeepSeekR1-Distill-70b-Q8_0.gguf [part1](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-70b.Q8_0.gguf.part1of2?download=true) [part2](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-70b.Q8_0.gguf.part2of2?download=true)| Q8_0 | 75.0GB | Best quality quantised version. | |
|
| [LlamaAligned-DeepSeekR1-Distill-70b-Q3_K_S.gguf](https://huggingface.co/BlueBeck/LlamaAligned-DeepSeekR1-Distill-70b/resolve/quants/LlamaAligned-DeepSeekR1-Distill-70b.Q3_K_S.gguf?download=true) | Q3_K_S | 30.9GB | Lower quality. | |
|
|
|
|
