Silicon-Alice-7B-GGUF

What's that?

Silicon-Alice-7B-GGUF is a quantized model based on Silicon-Masha-7B aiming to be both strong in RP, be smart and understand Russian, that can follow character maps very well. This model understands Russian better than the previous one. It is suitable for RP/ERP and general use. It can be run on weak samovar using llama.cpp or koboldcpp.

Prompt Template (Alpaca)

I found the best SillyTavern results from using the Noromaid template but please try other templates! Let me know if you find anything good.

SillyTavern config files: Context, Instruct.

Additionally, here is my highly recommended Text Completion preset. You can tweak this by adjusting temperature up or dropping min p to boost creativity or raise min p to increase stability. You shouldn't need to touch anything else!

Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Response:

Provided files

Name	Quant method	Bits	Use case
silicon-alice-7b.Q2_K.gguf	Q2_K	2	smallest, significant quality loss - not recommended for most purposes
silicon-alice-7b.Q3_K_M.gguf	Q3_K_M	3	very small, high quality loss
silicon-alice-7b.Q4_0.gguf	Q4_0	4	legacy; small, very high quality loss - prefer using Q3_K_M
silicon-alice-7b.Q4_K_M.gguf	Q4_K_M	4	medium, balanced quality - recommended
silicon-alice-7b.Q5_0.gguf	Q5_0	5	legacy; medium, balanced quality - prefer using Q4_K_M
silicon-alice-7b.Q5_K_M.gguf	Q5_K_M	5	large, very low quality loss - recommended
silicon-alice-7b.Q6_K.gguf	Q6_K	6	very large, extremely low quality loss
silicon-alice-7b.Q8_0.gguf	Q8_0	8	very large, extremely low quality loss - not recommended

How run it ?

llama.cpp

./main -ngl 35 -m silicon-alice-7b.Q4_K_M.gguf --color -c 32768 --temp 0.4 --repeat_penalty 1.1 -n -1 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n{system_message}\n### Instruction:{prompt}\n### Response:\n"