---
base_model:
- deepseek-ai/DeepSeek-V3.1-Base
base_model_relation: quantized
---

## Q4_K_M static quant of deepseek-ai/DeepSeek-V3.1-Base

Using <a href="https://github.com/ggml-org/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggml-org/llama.cpp/releases/tag/b6182">b6182</a> for quantization.

Original model: https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Base

Uploading this since I'm using it to calculate imatrix, figured might as well provide it in the meantime

Remember, this is a **BASE** model, so it likely will not chat properly unless you give it multiple turns of examples, for instance I've had success with:

```
./llama-cli -m /models/deepseek-ai_DeepSeek-V3.1-Base-Q4_K_M-00001-of-00011.gguf -p "You are a helpful assistant.<User>Hello, who are you?<Assistant>I am DeepSeek, a helpful AI assistant.<User>How are you today?<Assistant>I'm doing well! Is there anything I can assist you with?<User>Can you explain the laws of thermodynamics?<Assistant>" -no-cnv -ngl 0 --reverse-prompt "<User>"
```

Prompt for easier viewing:

`You are a helpful assistant.<User>Hello, who are you?<Assistant>I am DeepSeek, a helpful AI assistant.<User>How are you today?<Assistant>I'm doing well! Is there anything I can assist you with?<User>Can you explain the laws of thermodynamics?<Assistant>" -no-cnv -ngl 0 --reverse-prompt "<User>"`

*Yes*, I am using `<User>` and `<Assistant>` as opposed to the special tokens `<｜User｜>` and `<｜Assistant｜>`, for some reason this seems to be more stable?

This resulted in a completely coherent reply:

> Sure, here's a brief explanation of the laws of thermodynamics: 1. Zeroth Law of Thermodynamics: If two thermodynamic systems are each in thermal equilibrium with a third system, then they are in thermal equilibrium with each other. 2. First Law of Thermodynamics: The total energy of an isolated system is constant; energy can be transformed from one form to another, but cannot be created or destroyed. 3. Second Law of Thermodynamics: The entropy of an isolated system not in equilibrium will tend to increase over time, approaching a maximum value at equilibrium. 4. Third Law of Thermodynamics: As the temperature of a system approaches absolute zero, the entropy of the system approaches a minimum value. Would you like more details on any of these laws?

The idea is that you need to teach the base model what a conversation looks like first, base models aren't usually capable of one-shotting a conversation since it hasn't been tuned to understand roles.

382G total size