|
--- |
|
license: cc-by-4.0 |
|
--- |
|
The [GGML model](https://huggingface.co/tpmccallum/llama-2-13b-deep-haiku-GGML/blob/main/llama-2-13b-deep-haiku.ggml.fp16.bin) contained herein was generated by following the successive process in the Collab environments at https://github.com/robgon-art/DeepHaiku-LLaMa. |
|
|
|
> **NOTE:** This tutorial uses [Meta AI's Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) as the starting point (before the above quantizing process is performed). Therefore you will need to visit [Meta's Llama webpage](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) and agree to Meta's License, Acceptable Use Policy, and to Meta’s privacy policy before fetching and using Llama models. |
|
|
|
> **TIP:** The only change (aside from adding usernames and access tokens) was to substitute the following code (note the downgrading of `llama.cpp` using `checkout cf348a6` below; this is so that the Collab created the older GGML version of the model instead of the newer GGUF version) in the [2_Deep_Haiku_Quantize_Model_to_GGML](https://github.com/robgon-art/DeepHaiku-LLaMa/blob/main/2_Deep_Haiku_Quantize_Model_to_GGML.ipynb) part of the aforementioned process: |
|
|
|
``` |
|
!rm -rf llama.cpp |
|
!git clone https://github.com/ggerganov/llama.cpp |
|
!cd llama.cpp && git pull && git checkout cf348a6 && make clean && LLAMA_CUBLAS=1 make |
|
!pip install numpy==1.23 |
|
``` |
