Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ inference: false
|
|
9 |
|
10 |
# Wizard-Vicuna-13B-GPTQ
|
11 |
|
12 |
-
This repo contains 4bit GPTQ format quantised models of [
|
13 |
|
14 |
It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
15 |
|
@@ -17,7 +17,8 @@ It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com
|
|
17 |
|
18 |
* [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ).
|
19 |
* [4bit and 5bit GGML models for CPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML).
|
20 |
-
|
|
|
21 |
## How to easily download and use this model in text-generation-webui
|
22 |
|
23 |
Open the text-generation-webui UI as normal.
|
@@ -53,7 +54,9 @@ It was created without the `--act-order` parameter. It may have slightly lower i
|
|
53 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py wizard-vicuna-13B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors wizard-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors
|
54 |
```
|
55 |
|
56 |
-
# Original
|
|
|
|
|
57 |
|
58 |
# WizardVicunaLM
|
59 |
### Wizard's dataset + ChatGPT's conversation extension + Vicuna's tuning method
|
|
|
9 |
|
10 |
# Wizard-Vicuna-13B-GPTQ
|
11 |
|
12 |
+
This repo contains 4bit GPTQ format quantised models of [junelee's wizard-vicuna 13B](https://huggingface.co/junelee/wizard-vicuna-13b).
|
13 |
|
14 |
It is the result of quantising to 4bit using [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa).
|
15 |
|
|
|
17 |
|
18 |
* [4bit GPTQ models for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GPTQ).
|
19 |
* [4bit and 5bit GGML models for CPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-GGML).
|
20 |
+
* [float16 HF format model for GPU inference](https://huggingface.co/TheBloke/wizard-vicuna-13B-HF).
|
21 |
+
|
22 |
## How to easily download and use this model in text-generation-webui
|
23 |
|
24 |
Open the text-generation-webui UI as normal.
|
|
|
54 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py wizard-vicuna-13B-HF c4 --wbits 4 --true-sequential --groupsize 128 --save_safetensors wizard-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors
|
55 |
```
|
56 |
|
57 |
+
# Original WizardVicuna-13B model card
|
58 |
+
|
59 |
+
Github page: https://github.com/melodysdreamj/WizardVicunaLM
|
60 |
|
61 |
# WizardVicunaLM
|
62 |
### Wizard's dataset + ChatGPT's conversation extension + Vicuna's tuning method
|