## Requirements
We recommend using the Python version of [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and installing it with the following command:
```bash
!pip install https://github.com/abetlen/llama-cpp-python/releases/download/v0.2.78/llama_cpp_python-0.2.78-cp310-cp310-linux_x86_64.whl
```
## How to use
Instead of cloning the repository, which may be inefficient, you can manually download the required GGUF file or use `huggingface-cli` (`pip install huggingface_hub`) as demonstrated below:
```bash
!huggingface-cli login --token $HUGGING_FACE_HUB_TOKEN
!huggingface-cli download PartAI/Dorna-Llama3-8B-Instruct-GGUF dorna-llama3-8b-instruct.Q8_0.gguf --local-dir . --local-dir-use-symlinks False
```
```Python
from llama_cpp import Llama
llm = Llama(
model_path="dorna-llama3-8b-instruct.Q8_0.gguf",
chat_format="llama-3",
n_gpu_layers=-1,
n_ctx=2048,
)
messages = [
{"role": "system", "content": "You are a helpful Persian assistant. Please answer questions in the asked language."},
{"role": "user", "content": "کاغذ A4 بزرگ تر است یا A5؟"},
]
result = llm.create_chat_completion(
messages = messages,
top_p=0.85,
temperature=0.1
)
print(result)
```
## Contact us
If you have any questions regarding this model, you can reach us via the [community](https://huggingface.co/PartAI/Dorna-Llama3-8B-Instruct-GGUF/discussions) on Hugging Face.