Hugging Face's logo

ArtusDev
/
TheDrummer_Cydonia-R1-24B-v4.1-EXL3

exl3
Model card Files
xet
Community
1
ArtusDev commited on
Commit
b541675
·
verified ·
1 Parent(s): c793901

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +236 -27
README.md CHANGED
@@ -5,37 +5,246 @@ quantized_by: ArtusDev
5
  tags:
6
  - exl3
7
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
- ## EXL3 Quants of TheDrummer/Cydonia-R1-24B-v4.1
10
 
11
- EXL3 quants of [TheDrummer/Cydonia-R1-24B-v4.1](https://huggingface.co/TheDrummer/Cydonia-R1-24B-v4.1) using <a href="https://github.com/turboderp-org/exllamav3/">exllamav3</a> for quantization.
 
 
 
 
 
12
 
13
- ### Quants
14
- | Quant(Revision) | Bits per Weight | Head Bits |
15
- | -------- | ---------- | --------- |
16
- | [2.5_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/2.5bpw_H6) | 2.5 | 6 |
17
- | [3.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.0bpw_H6) | 3.0 | 6 |
18
- | [3.5_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.5bpw_H6) | 3.5 | 6 |
19
- | [4.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.0bpw_H6) | 4.0 | 6 |
20
- | [4.5_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.5bpw_H6) | 4.5 | 6 |
21
- | [5.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/5.0bpw_H6) | 5.0 | 6 |
22
- | [6.0_H6](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/6.0bpw_H6) | 6.0 | 6 |
23
- | [8.0_H8](https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/8.0bpw_H8) | 8.0 | 8 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
- ### Downloading quants with huggingface-cli
 
 
 
 
 
 
 
 
 
 
 
 
 
26
 
27
- <details>
28
- <summary>Click to view download instructions</summary>
 
 
 
 
 
 
 
29
 
30
- Install hugginface-cli:
 
 
 
 
 
31
 
32
- ```bash
33
- pip install -U "huggingface_hub[cli]"
34
- ```
35
-
36
- Download quant by targeting the specific quant revision (branch):
37
-
38
- ```
39
- huggingface-cli download ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3 --revision "5.0bpw_H6" --local-dir ./
40
- ```
41
- </details>
 
5
  tags:
6
  - exl3
7
  ---
8
+ <style>
9
+ .container-dark {
10
+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
11
+ line-height: 1.6;
12
+ color: #d4d4d4;
13
+ }
14
+ a {
15
+ color: #569cd6;
16
+ text-decoration: none;
17
+ font-weight: 600;
18
+ }
19
+ a:hover {
20
+ text-decoration: underline;
21
+ }
22
+ .card-dark {
23
+ background-color: #252526;
24
+ border-radius: 12px;
25
+ padding: 24px;
26
+ margin-bottom: 20px;
27
+ box-shadow: 0 4px 12px rgba(0,0,0,0.3);
28
+ border: 1px solid #3c3c3c;
29
+ }
30
+ .card-dark h1 {
31
+ font-size: 2.2em;
32
+ color: #ffffff;
33
+ text-align: center;
34
+ margin-bottom: 10px;
35
+ }
36
+ .card-dark.card-dark-title h1 {
37
+ font-size: 1.5em;
38
+ }
39
+ .card-dark .subtitle {
40
+ text-align: center;
41
+ font-size: 1.1em;
42
+ color: #a0a0a0;
43
+ }
44
+ .card-dark h2 {
45
+ font-size: 1.5em;
46
+ margin-top: 0;
47
+ padding-bottom: 10px;
48
+ border-bottom: 1px solid #3c3c3c;
49
+ color: #c586c0;
50
+ }
51
+ .card-dark h3 {
52
+ font-size: 1.2em;
53
+ color: #d4d4d4;
54
+ }
55
+ .styled-table {
56
+ display: table;
57
+ border: none;
58
+ width: 100%;
59
+ font-size: 0.95em;
60
+ margin-bottom: 0px;
61
+ }
62
+ .styled-table thead th {
63
+ background-color: #333333;
64
+ color: #c586c0;
65
+ text-align: left;
66
+ }
67
+ .styled-table th {
68
+ padding: 12px 15px;
69
+ }
70
+ .styled-table td {
71
+ padding: 0;
72
+ }
73
+ .styled-table table, .styled-table th, .styled-table td {
74
+ border-left: none;
75
+ border-right: none;
76
+ border-bottom: none;
77
+ }
78
+ .styled-table td {
79
+ border-bottom: 1px solid #3c3c3c;
80
+ }
81
+ .styled-table tbody tr {
82
+ transition: background-color 0.1s ease;
83
+ }
84
+ .styled-table tbody tr:hover {
85
+ background-color: #3a3a3a;
86
+ }
87
+ .styled-table tr:last-child td {
88
+ border-bottom: none;
89
+ }
90
+ .styled-table td a {
91
+ display: block;
92
+ padding: 12px 15px;
93
+ }
94
+ .styled-table td a.fake-link {
95
+ text-decoration:none;
96
+ color:inherit;
97
+ }
98
+ details {
99
+ margin-top: 20px;
100
+ border: 1px solid #3c3c3c;
101
+ border-radius: 8px;
102
+ overflow: hidden;
103
+ }
104
+ summary {
105
+ cursor: pointer;
106
+ padding: 12px 18px;
107
+ background-color: #6A5ACD;
108
+ font-weight: 600;
109
+ display: flex;
110
+ align-items: center;
111
+ gap: 10px;
112
+ justify-content: space-between;
113
+ list-style: none;
114
+ }
115
+ summary::-webkit-details-marker {
116
+ display: none;
117
+ }
118
+ summary:hover {
119
+ filter: brightness(1.1);
120
+ }
121
+ summary::after {
122
+ content: '';
123
+ display: inline-block;
124
+ width: 8px;
125
+ height: 8px;
126
+ border-bottom: 2px solid white;
127
+ border-right: 2px solid white;
128
+ transform: rotate(45deg);
129
+ transition: transform 0.3s ease;
130
+ }
131
+ details[open] > summary::after {
132
+ transform: rotate(225deg);
133
+ }
134
+ .details-content {
135
+ padding: 18px;
136
+ }
137
+ .btn-purple {
138
+ display: inline-block;
139
+ background-color: #6A5ACD;
140
+ color: white !important;
141
+ padding: 12px 24px;
142
+ border-radius: 8px;
143
+ text-decoration: none;
144
+ font-weight: 600;
145
+ transition: background-color 0.3s ease, transform 0.2s ease;
146
+ text-align: center;
147
+ }
148
+ .btn-purple:hover {
149
+ background-color: #7B68EE;
150
+ transform: translateY(-2px);
151
+ }
152
+ </style>
153
 
154
+ <div class="container-dark">
155
 
156
+ <div class="card-dark card-dark-title">
157
+ <h1>ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3</h1>
158
+ <p class="subtitle">
159
+ EXL3 quants of <a href="https://huggingface.co/TheDrummer/Cydonia-R1-24B-v4.1" target="_blank">TheDrummer/Cydonia-R1-24B-v4.1</a> using <a href="https://github.com/turboderp-org/exllamav3/" target="_blank">exllamav3</a> for quantization.
160
+ </p>
161
+ </div>
162
 
163
+ <div class="card-dark">
164
+ <h2>Quants</h2>
165
+ <table class="styled-table">
166
+ <thead>
167
+ <tr>
168
+ <th>Quant</th>
169
+ <th>BPW</th>
170
+ <th>Head Bits</th>
171
+ </tr>
172
+ </thead>
173
+ <tbody>
174
+ <tr>
175
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/2.5bpw_H6" target="_blank">2.5_H6</a></td>
176
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/2.5bpw_H6" target="_blank">2.5</a></td>
177
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/2.5bpw_H6" target="_blank">6</a></td>
178
+ </tr>
179
+ <tr>
180
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.0bpw_H6" target="_blank">3.0_H6</a></td>
181
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.0bpw_H6" target="_blank">3.0</a></td>
182
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.0bpw_H6" target="_blank">6</a></td>
183
+ </tr>
184
+ <tr>
185
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.5bpw_H6" target="_blank">3.5_H6</a></td>
186
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.5bpw_H6" target="_blank">3.5</a></td>
187
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/3.5bpw_H6" target="_blank">6</a></td>
188
+ </tr>
189
+ <tr>
190
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.0bpw_H6" target="_blank">4.0_H6</a></td>
191
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.0bpw_H6" target="_blank">4.0</a></td>
192
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.0bpw_H6" target="_blank">6</a></td>
193
+ </tr>
194
+ <tr>
195
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.5bpw_H6" target="_blank">4.5_H6</a></td>
196
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.5bpw_H6" target="_blank">4.5</a></td>
197
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/4.5bpw_H6" target="_blank">6</a></td>
198
+ </tr>
199
+ <tr>
200
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/5.0bpw_H6" target="_blank">5.0_H6</a></td>
201
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/5.0bpw_H6" target="_blank">5.0</a></td>
202
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/5.0bpw_H6" target="_blank">6</a></td>
203
+ </tr>
204
+ <tr>
205
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/6.0bpw_H6" target="_blank">6.0_H6</a></td>
206
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/6.0bpw_H6" target="_blank">6.0</a></td>
207
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/6.0bpw_H6" target="_blank">6</a></td>
208
+ </tr>
209
+ <tr>
210
+ <td><a href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/8.0bpw_H8" target="_blank">8.0_H8</a></td>
211
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/8.0bpw_H8" target="_blank">8.0</a></td>
212
+ <td><a class="fake-link" href="https://huggingface.co/ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3/tree/8.0bpw_H8" target="_blank">8</a></td>
213
+ </tr>
214
+ </tbody>
215
+ </table>
216
+ </div>
217
 
218
+ <div class="card-dark">
219
+ <h2>How to Download and Use Quants</h2>
220
+ <p>You can download quants by targeting specific size using the Hugging Face CLI.</p>
221
+ <details>
222
+ <summary>Click for download commands</summary>
223
+ <div class="details-content">
224
+ <b>1. Install huggingface-cli:</b>
225
+ <pre><code>pip install -U "huggingface_hub[cli]"</code></pre>
226
+ <b>2. Download a specific quant:</b>
227
+ <pre><code>huggingface-cli download ArtusDev/TheDrummer_Cydonia-R1-24B-v4.1-EXL3 --revision "5.0bpw_H6" --local-dir ./</code></pre>
228
+ </div>
229
+ </details>
230
+ <p>EXL3 quants can be run with any inference client that supports EXL3, such as <a href="https://github.com/theroyallab/tabbyapi" target="_blank"><b>TabbyAPI</b></a>. Refer to <a href="https://github.com/theroyallab/tabbyAPI/wiki/01.-Getting-Started" target="_blank">documentation</a> for set up instructions.</p>
231
+ </div>
232
 
233
+ <div class="card-dark">
234
+ <h2>Quant Requests</h2>
235
+ <div style="text-align: center; margin-top: 25px;">
236
+ <a href="https://huggingface.co/ArtusDev/requests-exl/discussions/new?title=[MODEL_NAME_HERE]&description=[MODEL_HF_LINK_HERE]" class="btn-purple" target="_blank">Request EXL3 Quants</a>
237
+ </div>
238
+ <p class="subtitle">
239
+ See <a href="https://huggingface.co/ArtusDev/requests-exl" target="_blank">EXL community hub</a> for request guidelines.
240
+ </p>
241
+ </div>
242
 
243
+ <div class="card-dark">
244
+ <h2>Acknowledgements</h2>
245
+ <p style="text-align: center; font-size: 1.1em; margin-top: 10px;">
246
+ Made possible with cloud compute from <a href="https://lium.io/" target="_blank"><b>lium.io</b></a>
247
+ </p>
248
+ </div>
249
 
250
+ </div>