Update README.md
Browse files
README.md
CHANGED
@@ -20,8 +20,8 @@ Model creator: [IBM Research](https://huggingface.co/ibm-granite)
|
|
20 |
[8bpw h8](https://huggingface.co/cgus/granite-guardian-3.1-2b-exl2/tree/8bpw-h8)
|
21 |
|
22 |
# Quantization notes
|
23 |
-
Made with exllamav2 0.2.7 with the default dataset.
|
24 |
-
|
25 |
Exl2 models require Nvidia RTX on Windows or Nvidia RTX or AMD ROCm GPUs on Linux.
|
26 |
|
27 |
# Granite Guardian 3.1 2B
|
|
|
20 |
[8bpw h8](https://huggingface.co/cgus/granite-guardian-3.1-2b-exl2/tree/8bpw-h8)
|
21 |
|
22 |
# Quantization notes
|
23 |
+
Made with exllamav2 0.2.7 with the default dataset. Granite3 models require 0.2.7 or newer version.
|
24 |
+
Exl2 models can be used with TabbyAPI, Text-Generation-WebUI and some others.
|
25 |
Exl2 models require Nvidia RTX on Windows or Nvidia RTX or AMD ROCm GPUs on Linux.
|
26 |
|
27 |
# Granite Guardian 3.1 2B
|