Hugging Face's logo

grimulkan
/
story-reverse-prompt-70b-rope8-32K-6bpw_h6_exl2

Text Generation
Transformers
llama
Model card Files Community
grimulkan commited on
Commit
fcc2a17
·
verified ·
1 Parent(s): d2bde79

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -1,3 +1,7 @@
1
  ---
2
  license: llama2
3
  ---
 
 
 
 
 
1
  ---
2
  license: llama2
3
  ---
4
+
5
+ This is a 6-bit quantization of [Story Reverse Prompt 70B 32K](https://huggingface.co/grimulkan/story-reverse-prompt-70b-rope8-32K-fp16). See that page for more details.
6
+
7
+ This quantization fits in 48GB+24GB (36/24 split) or 3x24GB (16/17/20 split) using Exllamav2 @ 32k context.