---
base_model:
- Almawave/Velvet-2B
base_model_relation: quantized
---
## DESCRIPTION

Velvet-2B converted to GGUF format (F32) with <a href="https://github.com/fbuciuni90/llama.cpp">fbuciuni90/llama.cpp</a> fork and quantized with <a href="https://github.com/ggerganov/llama.cpp">ggerganov/llama.cpp</a> commit b4689.

**NOTE: The Velvet tokenizer is not yet compatible with ggerganov/llama.cpp.** Please wait for pull request <a href="https://github.com/ggerganov/llama.cpp/pull/11716">#11716</a> to be merged, or compile it yourself.

Original Model: https://huggingface.co/Almawave/Velvet-2B

## PROMPT FORMAT

Basic prompt format:

```
<s><instruction>{prompt}</instruction>
```

Prompt format with system message:

```
<s><instruction>{system_prompt}\n\n{prompt}</instruction>
```

## DOWNLOAD

| Quant | Link |
| ----- | ---- |
| Q3_K_S | [Velvet-2B-Q3_K_S.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q3_K_S.gguf) |
| Q3_K_M | [Velvet-2B-Q3_K_M.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q3_K_M.gguf) |
| Q4_K_S | [Velvet-2B-Q4_K_S.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q4_K_S.gguf) |
| Q4_K_M | [Velvet-2B-Q4_K_M.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q4_K_M.gguf) |
| Q5_K_S | [Velvet-2B-Q5_K_S.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q5_K_S.gguf) |
| Q5_K_M | [Velvet-2B-Q5_K_M.gguf](https://huggingface.co/DagMeow/Velvet-2B-GGUF/blob/main/Velvet-2B-Q5_K_M.gguf) |

## BYE :3