Upload README.md
Browse files
README.md
CHANGED
|
@@ -29,6 +29,11 @@ Original model: https://huggingface.co/Deci/DeciLM-7B-Instruct
|
|
| 29 |
```
|
| 30 |
|
| 31 |
Modified llama.cpp to support DeciLMCausalModel's variable Grouped Query Attention.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
## Download a file (not the whole branch) from below:
|
| 34 |
|
|
|
|
| 29 |
```
|
| 30 |
|
| 31 |
Modified llama.cpp to support DeciLMCausalModel's variable Grouped Query Attention.
|
| 32 |
+
Please note that the HF model of Deci-7B-Instruct uses dynamic NTK-ware RoPE scaling.
|
| 33 |
+
However, llama.cpp doesn't support it yet, so my modifification also just ignore
|
| 34 |
+
the dynamic NTK-ware RoPE scaling settings in the config.json. Since the ggufs seem
|
| 35 |
+
working, for the time being just use them as is until I figure out how to implement
|
| 36 |
+
dynamic NTK-ware RoPE scaling.
|
| 37 |
|
| 38 |
## Download a file (not the whole branch) from below:
|
| 39 |
|