secretmoon commited on
Commit
46bad78
1 Parent(s): 452d6c6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -16,7 +16,7 @@ pipeline_tag: text-generation
16
 
17
  ## About
18
 
19
- GGUF imatrix quants of **[AlexBefest/WoonaV1.2-9b](https://huggingface.co/AlexBefest/WoonaV1.2-9b)** model. All quants, except of q6_k and q8_0 was maded with imatrix quantization method.
20
 
21
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6336c5b3e3ac69e6a90581da/1KKzl7nz9EyWI4CLvBvPp.png)
22
 
@@ -32,15 +32,15 @@ GGUF imatrix quants of **[AlexBefest/WoonaV1.2-9b](https://huggingface.co/AlexBe
32
 
33
  | Name | Quant method | Bits | Size | Min RAM required | Use case |
34
  | ---- | ---- | ---- | ---- | ---- | ----- |
35
- | [WoonaV1.2-9b-imat-Q2_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q2_K.gguf) | Q2_K [imatrix] | 2 | 3.5 GB| 5.1 GB | very, significant quality loss - not recommended, but usable (faster) |
36
  | [WoonaV1.2-9b-imat-IQ3_XXS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_XXS.gguf) | IQ3_XXS [imatrix] | 3 | 3.5 GB| 5.1 GB | small, high quality loss |
37
  | [WoonaV1.2-9b-imat-IQ3_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_M.gguf) | IQ3_M [imatrix] | 3 | 4.2 GB| 5.7 GB | small, high quality loss |
38
- | [WoonaV1.2-9b-imat-IQ4_XS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ4_XS.gguf) | Q4_XS [imatrix] | 4 | 4.8 GB| 6.3 GB | medium, substantial quality loss |
39
  | [WoonaV1.2-9b-imat-Q4_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_S.gguf) | Q4_K_S [imatrix] | 4 | 5.1 GB| 6.7 GB | medium, balanced quality loss |
40
  | [WoonaV1.2-9b-imat-Q4_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_M.gguf) | Q4_K_M [imatrix] | 4 | 5.4 GB| 6.9 GB | medium, balanced quality - recommended |
41
  | [WoonaV1.2-9b-imat-Q5_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_S.gguf) | Q5_K_S [imatrix] | 5 | 6 GB| 7.6 GB | large, low quality loss - recommended |
42
  | [WoonaV1.2-9b-imat-Q5_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_M.gguf) | Q5_K_M [imatrix] | 5 | 6.2 GB| 7.8 GB | large, very low quality loss - recommended |
43
- | [WoonaV1.2-9b-Q6_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q6_K [static] | 6 | 7.1 GB| 8.7 GB | very large, near perfect loss - recommended |
44
  | [WoonaV1.2-9b-Q8_0.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q8_0 [static] | 8 | 9.2 GB| 10.8 GB | very large, extremely low quality loss
45
 
46
 
 
16
 
17
  ## About
18
 
19
+ GGUF imatrix quants of **[AlexBefest/WoonaV1.2-9b](https://huggingface.co/AlexBefest/WoonaV1.2-9b)** model. All quants, except Q6_k and Q8_0 was maded with imatrix quantization method.
20
 
21
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6336c5b3e3ac69e6a90581da/1KKzl7nz9EyWI4CLvBvPp.png)
22
 
 
32
 
33
  | Name | Quant method | Bits | Size | Min RAM required | Use case |
34
  | ---- | ---- | ---- | ---- | ---- | ----- |
35
+ | [WoonaV1.2-9b-imat-Q2_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q2_K.gguf) | Q2_K [imatrix] | 2 | 3.5 GB| 5.1 GB | small, very high quality loss - not recommended, but usable (probably faster than Q3_XXS, but worse) |
36
  | [WoonaV1.2-9b-imat-IQ3_XXS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_XXS.gguf) | IQ3_XXS [imatrix] | 3 | 3.5 GB| 5.1 GB | small, high quality loss |
37
  | [WoonaV1.2-9b-imat-IQ3_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ3_M.gguf) | IQ3_M [imatrix] | 3 | 4.2 GB| 5.7 GB | small, high quality loss |
38
+ | [WoonaV1.2-9b-imat-IQ4_XS.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-IQ4_XS.gguf) | IQ4_XS [imatrix] | 4 | 4.8 GB| 6.3 GB | medium, slightly worse than Q4_K_M|
39
  | [WoonaV1.2-9b-imat-Q4_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_S.gguf) | Q4_K_S [imatrix] | 4 | 5.1 GB| 6.7 GB | medium, balanced quality loss |
40
  | [WoonaV1.2-9b-imat-Q4_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q4_K_M.gguf) | Q4_K_M [imatrix] | 4 | 5.4 GB| 6.9 GB | medium, balanced quality - recommended |
41
  | [WoonaV1.2-9b-imat-Q5_K_S.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_S.gguf) | Q5_K_S [imatrix] | 5 | 6 GB| 7.6 GB | large, low quality loss - recommended |
42
  | [WoonaV1.2-9b-imat-Q5_K_M.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-imat-Q5_K_M.gguf) | Q5_K_M [imatrix] | 5 | 6.2 GB| 7.8 GB | large, very low quality loss - recommended |
43
+ | [WoonaV1.2-9b-Q6_K.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q6_K [static] | 6 | 7.1 GB| 8.7 GB | very large, near perfect quality - recommended |
44
  | [WoonaV1.2-9b-Q8_0.gguf](https://huggingface.co/secretmoon/WoonaV1.2-9b-GGUF-Imatrix/blob/main/WoonaV1.2-9b-Q6_K.gguf) | Q8_0 [static] | 8 | 9.2 GB| 10.8 GB | very large, extremely low quality loss
45
 
46