Hugging Face's logo

unsloth
/
Kimi-K2-Instruct-BF16

Text Generation
Transformers
Safetensors
deepseek_v3
unsloth
conversational
custom_code
text-generation-inference
Model card Files
xet
Community
1
danielhanchen commited on
Commit
b55a352
·
verified ·
1 Parent(s): b977723

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .gitattributes +3 -0
  2. LICENSE +27 -0
  3. README.md +857 -0
  4. chat_template.jinja +37 -0
  5. config.json +72 -0
  6. configuration_deepseek.py +212 -0
  7. generation_config.json +4 -0
  8. kimi-logo.png +0 -0
  9. model-1-of-61.safetensors +3 -0
  10. model-10-of-61.safetensors +3 -0
  11. model-11-of-61.safetensors +3 -0
  12. model-12-of-61.safetensors +3 -0
  13. model-13-of-61.safetensors +3 -0
  14. model-14-of-61.safetensors +3 -0
  15. model-15-of-61.safetensors +3 -0
  16. model-16-of-61.safetensors +3 -0
  17. model-17-of-61.safetensors +3 -0
  18. model-18-of-61.safetensors +3 -0
  19. model-19-of-61.safetensors +3 -0
  20. model-2-of-61.safetensors +3 -0
  21. model-20-of-61.safetensors +3 -0
  22. model-21-of-61.safetensors +3 -0
  23. model-22-of-61.safetensors +3 -0
  24. model-23-of-61.safetensors +3 -0
  25. model-24-of-61.safetensors +3 -0
  26. model-25-of-61.safetensors +3 -0
  27. model-26-of-61.safetensors +3 -0
  28. model-27-of-61.safetensors +3 -0
  29. model-28-of-61.safetensors +3 -0
  30. model-29-of-61.safetensors +3 -0
  31. model-3-of-61.safetensors +3 -0
  32. model-30-of-61.safetensors +3 -0
  33. model-31-of-61.safetensors +3 -0
  34. model-32-of-61.safetensors +3 -0
  35. model-33-of-61.safetensors +3 -0
  36. model-34-of-61.safetensors +3 -0
  37. model-35-of-61.safetensors +3 -0
  38. model-36-of-61.safetensors +3 -0
  39. model-37-of-61.safetensors +3 -0
  40. model-38-of-61.safetensors +3 -0
  41. model-39-of-61.safetensors +3 -0
  42. model-4-of-61.safetensors +3 -0
  43. model-40-of-61.safetensors +3 -0
  44. model-41-of-61.safetensors +3 -0
  45. model-42-of-61.safetensors +3 -0
  46. model-43-of-61.safetensors +3 -0
  47. model-44-of-61.safetensors +3 -0
  48. model-45-of-61.safetensors +3 -0
  49. model-46-of-61.safetensors +3 -0
  50. model-47-of-61.safetensors +3 -0
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
37
+ figures/Base-Evaluation.png filter=lfs diff=lfs merge=lfs -text
38
+ banner.png filter=lfs diff=lfs merge=lfs -text
LICENSE ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Modified MIT License
2
+
3
+ Copyright (c) 2025 Moonshot AI
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the “Software”), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ Our only modification part is that, if the Software (or any derivative works
24
+ thereof) is used for any of your commercial products or services that have
25
+ more than 100 million monthly active users, or more than 20 million US dollars
26
+ (or equivalent in other currencies) in monthly revenue, you shall prominently
27
+ display "Kimi K2" on the user interface of such product or service.
README.md ADDED
@@ -0,0 +1,857 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - unsloth
4
+ - unsloth
5
+ - unsloth
6
+ base_model:
7
+ - moonshotai/Kimi-K2-Instruct-BF16
8
+ license: other
9
+ license_name: modified-mit
10
+ library_name: transformers
11
+ ---
12
+ > [!NOTE]
13
+ > Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
14
+ >
15
+
16
+ <div>
17
+ <p style="margin-top: 0;margin-bottom: 0;">
18
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
19
+ </p>
20
+ <div style="display: flex; gap: 5px; align-items: center; ">
21
+ <a href="https://github.com/unslothai/unsloth/">
22
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
23
+ </a>
24
+ <a href="https://discord.gg/unsloth">
25
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
26
+ </a>
27
+ <a href="https://docs.unsloth.ai/">
28
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
29
+ </a>
30
+ </div>
31
+ </div>
32
+
33
+ > [!NOTE]
34
+ > Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
35
+ >
36
+
37
+ <div>
38
+ <p style="margin-top: 0;margin-bottom: 0;">
39
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
40
+ </p>
41
+ <div style="display: flex; gap: 5px; align-items: center; ">
42
+ <a href="https://github.com/unslothai/unsloth/">
43
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
44
+ </a>
45
+ <a href="https://discord.gg/unsloth">
46
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
47
+ </a>
48
+ <a href="https://docs.unsloth.ai/">
49
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
50
+ </a>
51
+ </div>
52
+ </div>
53
+
54
+ > [!NOTE]
55
+ > Includes our **chat template fixes**! <br> For `llama.cpp`, use `--jinja`
56
+ >
57
+
58
+ <div>
59
+ <p style="margin-top: 0;margin-bottom: 0;">
60
+ <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
61
+ </p>
62
+ <div style="display: flex; gap: 5px; align-items: center; ">
63
+ <a href="https://github.com/unslothai/unsloth/">
64
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
65
+ </a>
66
+ <a href="https://discord.gg/unsloth">
67
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
68
+ </a>
69
+ <a href="https://docs.unsloth.ai/">
70
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
71
+ </a>
72
+ </div>
73
+ </div>
74
+
75
+ <div align="center">
76
+ <picture>
77
+ <img src="figures/kimi-logo.png" width="30%" alt="Kimi K2: Open Agentic Intellignece">
78
+ </picture>
79
+ </div>
80
+
81
+ <hr>
82
+
83
+ <div align="center" style="line-height:1">
84
+ <a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K2-ff6b6b?color=1783ff&logoColor=white"/></a>
85
+ <a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
86
+ </div>
87
+
88
+ <div align="center" style="line-height: 1;">
89
+ <a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
90
+ <a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
91
+ <a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
92
+ </div>
93
+
94
+ <div align="center" style="line-height: 1;">
95
+ <a href="https://github.com/moonshotai/Kimi-K2/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>
96
+ </div>
97
+
98
+ <p align="center">
99
+ <b>📰&nbsp;&nbsp;<a href="https://moonshotai.github.io/Kimi-K2/">Tech Blog</a></b> &nbsp;&nbsp;&nbsp; | &nbsp;&nbsp;&nbsp; <b>📄&nbsp;&nbsp;Paper Link (comming soon)</b>
100
+ </p>
101
+
102
+ ## 1. Model Introduction
103
+
104
+ Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model with 32 billion activated parameters and 1 trillion total parameters. Trained with the Muon optimizer, Kimi K2 achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being meticulously optimized for agentic capabilities.
105
+
106
+ ### Key Features
107
+ - Large-Scale Training: Pre-trained a 1T parameter MoE model on 15.5T tokens with zero training instability.
108
+ - MuonClip Optimizer: We apply the Muon optimizer to an unprecedented scale, and develop novel optimization techniques to resolve instabilities while scaling up.
109
+ - Agentic Intelligence: Specifically designed for tool use, reasoning, and autonomous problem-solving.
110
+
111
+ ### Model Variants
112
+ - **Kimi-K2-Base**: The foundation model, a strong start for researchers and builders who want full control for fine-tuning and custom solutions.
113
+ - **Kimi-K2-Instruct**: The post-trained model best for drop-in, general-purpose chat and agentic experiences. It is a reflex-grade model without long thinking.
114
+
115
+ <div align="center">
116
+ <picture>
117
+ <img src="figures/banner.png" width="80%" alt="Evaluation Results">
118
+ </picture>
119
+ </div>
120
+
121
+ ## 2. Model Summary
122
+
123
+ <div align="center">
124
+
125
+
126
+ | | |
127
+ |:---:|:---:|
128
+ | **Architecture** | Mixture-of-Experts (MoE) |
129
+ | **Total Parameters** | 1T |
130
+ | **Activated Parameters** | 32B |
131
+ | **Number of Layers** (Dense layer included) | 61 |
132
+ | **Number of Dense Layers** | 1 |
133
+ | **Attention Hidden Dimension** | 7168 |
134
+ | **MoE Hidden Dimension** (per Expert) | 2048 |
135
+ | **Number of Attention Heads** | 64 |
136
+ | **Number of Experts** | 384 |
137
+ | **Selected Experts per Token** | 8 |
138
+ | **Number of Shared Experts** | 1 |
139
+ | **Vocabulary Size** | 160K |
140
+ | **Context Length** | 128K |
141
+ | **Attention Mechanism** | MLA |
142
+ | **Activation Function** | SwiGLU |
143
+ </div>
144
+
145
+ ## 3. Evaluation Results
146
+
147
+ #### Instruction model evaluation results
148
+
149
+ <div align="center">
150
+ <table>
151
+ <thead>
152
+ <tr>
153
+ <th align="center">Benchmark</th>
154
+ <th align="center">Metric</th>
155
+ <th align="center"><sup>Kimi K2 Instruct</sup></th>
156
+ <th align="center"><sup>DeepSeek-V3-0324</sup></th>
157
+ <th align="center"><sup>Qwen3-235B-A22B <br><sup>(non-thinking)</sup></sup></th>
158
+ <th align="center"><sup>Claude Sonnet 4 <br><sup>(w/o extended thinking)</sup></sup></th>
159
+ <th align="center"><sup>Claude Opus 4 <br><sup>(w/o extended thinking)</sup></sup></th>
160
+ <th align="center"><sup>GPT-4.1</sup></th>
161
+ <th align="center"><sup>Gemini 2.5 Flash <br> Preview (05-20)</sup></th>
162
+ </tr>
163
+ </thead>
164
+ <tbody>
165
+ <tr>
166
+ <td align="center" colspan=9><strong>Coding Tasks</strong></td>
167
+ </tr>
168
+ <tr>
169
+ <td align="center">LiveCodeBench v6<br><sup>(Aug 24 - May 25)</sup></td>
170
+ <td align="center">Pass@1</td>
171
+ <td align="center"><strong>53.7</strong></td>
172
+ <td align="center">46.9</td>
173
+ <td align="center">37.0</td>
174
+ <td align="center">48.5</td>
175
+ <td align="center">47.4</td>
176
+ <td align="center">44.7</td>
177
+ <td align="center">44.7</td>
178
+ </tr>
179
+ <tr>
180
+ <td align="center">OJBench</td>
181
+ <td align="center">Pass@1</td>
182
+ <td align="center"><strong>27.1</strong></td>
183
+ <td align="center">24.0</td>
184
+ <td align="center">11.3</td>
185
+ <td align="center">15.3</td>
186
+ <td align="center">19.6</td>
187
+ <td align="center">19.5</td>
188
+ <td align="center">19.5</td>
189
+ </tr>
190
+
191
+ <tr>
192
+ <td align="center">MultiPL-E</td>
193
+ <td align="center">Pass@1</td>
194
+ <td align="center"><ins><strong>85.7</strong></ins></td>
195
+ <td align="center">83.1</td>
196
+ <td align="center">78.2</td>
197
+ <td align="center">88.6</td>
198
+ <td align="center"><strong>89.6</strong></td>
199
+ <td align="center">86.7</td>
200
+ <td align="center">85.6</td>
201
+ </tr>
202
+
203
+ <tr>
204
+ <td align="center">SWE-bench Verified <br/><sup>(Agentless Coding)</sup></td>
205
+ <td align="center">Single Patch w/o Test (Acc)</td>
206
+ <td align="center"><ins><strong>51.8</strong></ins></td>
207
+ <td align="center">36.6</td>
208
+ <td align="center">39.4</td>
209
+ <td align="center">50.2</td>
210
+ <td align="center"><strong>53.0</strong></td>
211
+ <td align="center">40.8</td>
212
+ <td align="center">32.6</td>
213
+ </tr>
214
+
215
+ <tr>
216
+ <td align="center" rowspan="2">SWE-bench Verified <br/> <sup>(Agentic Coding)</sup></td>
217
+ <td align="center">Single Attempt (Acc)</td>
218
+ <td align="center"><ins><strong>65.8</strong></ins></td>
219
+ <td align="center">38.8</td>
220
+ <td align="center">34.4</td>
221
+ <td align="center"><strong>72.7</strong><sup>*</sup></td>
222
+ <td align="center">72.5<sup>*</sup></td>
223
+ <td align="center">54.6</td>
224
+ <td align="center">—</td>
225
+ </tr>
226
+
227
+ <tr>
228
+ <!--<td align="center">(Agentic Coding)</td>-->
229
+ <td align="center">Multiple Attempts (Acc)</td>
230
+ <td align="center"><ins><strong>71.6</strong></ins></td>
231
+ <td align="center">—</td>
232
+ <td align="center">—</td>
233
+ <td align="center"><strong>80.2</strong></td>
234
+ <td align="center">79.4<sup>*</sup></td>
235
+ <td align="center">—</td>
236
+ <td align="center">—</td>
237
+ </tr>
238
+
239
+ <tr>
240
+ <td align="center">SWE-bench Multilingual<br /> <sup>(Agentic Coding)</sup></td>
241
+ <td align="center">Single Attempt (Acc)</td>
242
+ <td align="center"><ins><strong>47.3</strong> </ins></td>
243
+ <td align="center">25.8</td>
244
+ <td align="center">20.9</td>
245
+ <td align="center"><strong>51.0</strong></td>
246
+ <td align="center">—</td>
247
+ <td align="center">31.5</td>
248
+ <td align="center">—</td>
249
+ </tr>
250
+
251
+ <tr>
252
+ <td align="center" rowspan="2">TerminalBench</td>
253
+ <td align="center">Inhouse Framework (Acc)</td>
254
+ <td align="center"><ins><strong>30.0</strong></ins></td>
255
+ <td align="center">—</td>
256
+ <td align="center">—</td>
257
+ <td align="center">35.5</td>
258
+ <td align="center"><strong>43.2</strong></td>
259
+ <td align="center">8.3</td>
260
+ <td align="center">—</td>
261
+ </tr>
262
+
263
+ <tr>
264
+ <!--<td align="center">TerminalBench</td>-->
265
+ <td align="center">Terminus (Acc)</td>
266
+ <td align="center"><ins><strong>25.0</strong> </ins></td>
267
+ <td align="center">16.3</td>
268
+ <td align="center">6.6</td>
269
+ <td align="center">—</td>
270
+ <td align="center">—</td>
271
+ <td align="center"><strong>30.3</strong></td>
272
+ <td align="center">16.8</td>
273
+ </tr>
274
+ <tr>
275
+ <td align="center">Aider-Polyglot</td>
276
+ <td align="center">Acc</td>
277
+ <td align="center">60.0</td>
278
+ <td align="center">55.1</td>
279
+ <td align="center"><ins><strong>61.8</strong></ins></td>
280
+ <td align="center">56.4</td>
281
+ <td align="center"><strong>70.7</strong></td>
282
+ <td align="center">52.4</td>
283
+ <td align="center">44.0</td>
284
+ </tr>
285
+ <tr>
286
+ <td align="center" colspan=9><strong>Tool Use Tasks</strong></td>
287
+ </tr>
288
+ <tr>
289
+ <td align="center">Tau2 retail</td>
290
+ <td align="center">Avg@4</td>
291
+ <td align="center"><ins><strong>70.6</strong></ins></td>
292
+ <td align="center">69.1</td>
293
+ <td align="center">57.0</td>
294
+ <td align="center">75.0</td>
295
+ <td align="center"><strong>81.8</strong></td>
296
+ <td align="center">74.8</td>
297
+ <td align="center">64.3</td>
298
+ </tr>
299
+ <tr>
300
+ <td align="center">Tau2 airline</td>
301
+ <td align="center">Avg@4</td>
302
+ <td align="center"><ins><strong>56.5</strong></ins></td>
303
+ <td align="center">39.0</td>
304
+ <td align="center">26.5</td>
305
+ <td align="center">55.5</td>
306
+ <td align="center"><strong>60.0</strong></td>
307
+ <td align="center">54.5</td>
308
+ <td align="center">42.5</td>
309
+ </tr>
310
+ <tr>
311
+ <td align="center">Tau2 telecom</td>
312
+ <td align="center">Avg@4</td>
313
+ <td align="center"><strong>65.8</strong></td>
314
+ <td align="center">32.5</td>
315
+ <td align="center">22.1</td>
316
+ <td align="center">45.2</td>
317
+ <td align="center">57.0</td>
318
+ <td align="center">38.6</td>
319
+ <td align="center">16.9</td>
320
+ </tr>
321
+ <tr>
322
+ <td align="center">AceBench</td>
323
+ <td align="center">Acc</td>
324
+ <td align="center"><ins><strong>76.5</strong></ins></td>
325
+ <td align="center">72.7</td>
326
+ <td align="center">70.5</td>
327
+ <td align="center">76.2</td>
328
+ <td align="center">75.6</td>
329
+ <td align="center"><strong>80.1</strong></td>
330
+ <td align="center">74.5</td>
331
+ </tr>
332
+ <tr>
333
+ <td align="center" colspan=9><strong>Math &amp; STEM Tasks</strong></td>
334
+ </tr>
335
+ <tr>
336
+ <td align="center">AIME 2024</td>
337
+ <td align="center">Avg@64</td>
338
+ <td align="center"><strong>69.6</strong></td>
339
+ <td align="center">59.4<sup>*</sup></td>
340
+ <td align="center">40.1<sup>*</sup></td>
341
+ <td align="center">43.4</td>
342
+ <td align="center">48.2</td>
343
+ <td align="center">46.5</td>
344
+ <td align="center">61.3</td>
345
+ </tr>
346
+ <tr>
347
+ <td align="center">AIME 2025</td>
348
+ <td align="center">Avg@64</td>
349
+ <td align="center"><strong>49.5</strong></td>
350
+ <td align="center">46.7</td>
351
+ <td align="center">24.7<sup>*</sup></td>
352
+ <td align="center">33.1<sup>*</sup></td>
353
+ <td align="center">33.9<sup>*</sup></td>
354
+ <td align="center">37.0</td>
355
+ <td align="center">46.6</td>
356
+ </tr>
357
+ <tr>
358
+ <td align="center">MATH-500</td>
359
+ <td align="center">Acc</td>
360
+ <td align="center"><strong>97.4</strong></td>
361
+ <td align="center">94.0<sup>*</sup></td>
362
+ <td align="center">91.2<sup>*</sup></td>
363
+ <td align="center">94.0</td>
364
+ <td align="center">94.4</td>
365
+ <td align="center">92.4</td>
366
+ <td align="center">95.4</td>
367
+ </tr>
368
+ <tr>
369
+ <td align="center">HMMT 2025</td>
370
+ <td align="center">Avg@32</td>
371
+ <td align="center"><strong>38.8</strong></td>
372
+ <td align="center">27.5</td>
373
+ <td align="center">11.9</td>
374
+ <td align="center">15.9</td>
375
+ <td align="center">15.9</td>
376
+ <td align="center">19.4</td>
377
+ <td align="center">34.7</td>
378
+ </tr>
379
+ <tr>
380
+ <td align="center">CNMO 2024</td>
381
+ <td align="center">Avg@16</td>
382
+ <td align="center">74.3</td>
383
+ <td align="center"><ins><strong>74.7</strong></ins></td>
384
+ <td align="center">48.6</td>
385
+ <td align="center">60.4</td>
386
+ <td align="center">57.6</td>
387
+ <td align="center">56.6</td>
388
+ <td align="center"><strong>75.0</strong></td>
389
+ </tr>
390
+ <tr>
391
+ <td align="center">PolyMath-en</td>
392
+ <td align="center">Avg@4</td>
393
+ <td align="center"><strong>65.1</strong></td>
394
+ <td align="center">59.5</td>
395
+ <td align="center">51.9</td>
396
+ <td align="center">52.8</td>
397
+ <td align="center">49.8</td>
398
+ <td align="center">54.0</td>
399
+ <td align="center">49.9</td>
400
+ </tr>
401
+
402
+ <tr>
403
+ <td align="center">ZebraLogic</td>
404
+ <td align="center">Acc</td>
405
+ <td align="center"><strong>89.0</strong></td>
406
+ <td align="center">84.0</td>
407
+ <td align="center">37.7<sup>*</sup></td>
408
+ <td align="center">73.7</td>
409
+ <td align="center">59.3</td>
410
+ <td align="center">58.5</td>
411
+ <td align="center">57.9</td>
412
+ </tr>
413
+
414
+ <tr>
415
+ <td align="center">AutoLogi</td>
416
+ <td align="center">Acc</td>
417
+ <td align="center"><ins><strong>89.5</strong></ins></td>
418
+ <td align="center">88.9</td>
419
+ <td align="center">83.3</td>
420
+ <td align="center"><strong>89.8</strong></td>
421
+ <td align="center">86.1</td>
422
+ <td align="center">88.2</td>
423
+ <td align="center">84.1</td>
424
+ </tr>
425
+
426
+ <tr>
427
+ <td align="center">GPQA-Diamond</td>
428
+ <td align="center">Avg@8</td>
429
+ <td align="center"><strong>75.1</strong></td>
430
+ <td align="center">68.4<sup>*</sup></td>
431
+ <td align="center">62.9<sup>*</sup></td>
432
+ <td align="center">70.0<sup>*</sup></td>
433
+ <td align="center">74.9<sup>*</sup></td>
434
+ <td align="center">66.3</td>
435
+ <td align="center">68.2</td>
436
+ </tr>
437
+
438
+ <tr>
439
+ <td align="center">SuperGPQA</td>
440
+ <td align="center">Acc</td>
441
+ <td align="center"><strong>57.2</strong></td>
442
+ <td align="center">53.7</td>
443
+ <td align="center">50.2</td>
444
+ <td align="center">55.7</td>
445
+ <td align="center">56.5</td>
446
+ <td align="center">50.8</td>
447
+ <td align="center">49.6</td>
448
+ </tr>
449
+
450
+ <tr>
451
+ <td align="center">Humanity's Last Exam<br><sup>(Text Only)</sup></td>
452
+ <td align="center">-</td>
453
+ <td align="center">4.7</td>
454
+ <td align="center">5.2</td>
455
+ <td align="center"><ins><strong>5.7</strong></ins></td>
456
+ <td align="center">5.8</td>
457
+ <td align="center"><strong>7.1</strong></td>
458
+ <td align="center">3.7</td>
459
+ <td align="center">5.6</td>
460
+ </tr>
461
+
462
+ <tr>
463
+ <td align="center" colspan=9><strong>General Tasks</strong></td>
464
+ </tr>
465
+
466
+ <tr>
467
+ <td align="center">MMLU</td>
468
+ <td align="center">EM</td>
469
+ <td align="center"><ins><strong>89.5</strong></ins></td>
470
+ <td align="center">89.4</td>
471
+ <td align="center">87.0</td>
472
+ <td align="center">91.5</td>
473
+ <td align="center"><strong>92.9</strong></td>
474
+ <td align="center">90.4</td>
475
+ <td align="center">90.1</td>
476
+ </tr>
477
+
478
+ <tr>
479
+ <td align="center">MMLU-Redux</td>
480
+ <td align="center">EM</td>
481
+ <td align="center"><ins><strong>92.7</strong></ins></td>
482
+ <td align="center">90.5</td>
483
+ <td align="center">89.2</td>
484
+ <td align="center">93.6</td>
485
+ <td align="center"><strong>94.2</strong></td>
486
+ <td align="center">92.4</td>
487
+ <td align="center">90.6</td>
488
+ </tr>
489
+
490
+ <tr>
491
+ <td align="center">MMLU-Pro</td>
492
+ <td align="center">EM</td>
493
+ <td align="center">81.1</td>
494
+ <td align="center"><ins><strong>81.2</strong></ins><sup>*</sup></td>
495
+ <td align="center">77.3</td>
496
+ <td align="center">83.7</td>
497
+ <td align="center"><strong>86.6</strong></td>
498
+ <td align="center">81.8</td>
499
+ <td align="center">79.4</td>
500
+ </tr>
501
+
502
+ <tr>
503
+ <td align="center">IFEval</td>
504
+ <td align="center">Prompt Strict</td>
505
+ <td align="center"><strong>89.8</strong></td>
506
+ <td align="center">81.1</td>
507
+ <td align="center">83.2<sup>*</sup></td>
508
+ <td align="center">87.6</td>
509
+ <td align="center">87.4</td>
510
+ <td align="center">88.0</td>
511
+ <td align="center">84.3</td>
512
+ </tr>
513
+
514
+ <tr>
515
+ <td align="center">Multi-Challenge</td>
516
+ <td align="center">Acc</td>
517
+ <td align="center"><strong>54.1</strong></td>
518
+ <td align="center">31.4</td>
519
+ <td align="center">34.0</td>
520
+ <td align="center">46.8</td>
521
+ <td align="center">49.0</td>
522
+ <td align="center">36.4</td>
523
+ <td align="center">39.5</td>
524
+ </tr>
525
+
526
+ <tr>
527
+ <td align="center">SimpleQA</td>
528
+ <td align="center">Correct</td>
529
+ <td align="center"><ins><strong>31.0</strong></ins></td>
530
+ <td align="center">27.7</td>
531
+ <td align="center">13.2</td>
532
+ <td align="center">15.9</td>
533
+ <td align="center">22.8</td>
534
+ <td align="center"><strong>42.3</strong></td>
535
+ <td align="center">23.3</td>
536
+ </tr>
537
+
538
+ <tr>
539
+ <td align="center">Livebench</td>
540
+ <td align="center">Pass@1</td>
541
+ <td align="center"><strong>76.4</strong></td>
542
+ <td align="center">72.4</td>
543
+ <td align="center">67.6</td>
544
+ <td align="center">74.8</td>
545
+ <td align="center">74.6</td>
546
+ <td align="center">69.8</td>
547
+ <td align="center">67.8</td>
548
+ </tr>
549
+ </tbody>
550
+ </table>
551
+ </div>
552
+ <sup>
553
+ • Bold denotes global SOTA, and underlined denotes open-source SOTA.
554
+ </sup><br/><sup>
555
+ • Data points marked with * are taken directly from the model's tech report or blog.
556
+ </sup><br/><sup>
557
+ • All metrics, except for SWE-bench Verified (Agentless), are evaluated with an 8k output token length. SWE-bench Verified (Agentless) is limited to a 16k output token length.
558
+ </sup><br/><sup>
559
+ • Kimi K2 achieves 65.8% pass@1 on the SWE-bench Verified tests with bash/editor tools (single-attempt patches, no test-time compute). It also achieves a 47.3% pass@1 on the SWE-bench Multilingual tests under the same conditions. Additionally, we report results on SWE-bench Verified tests (71.6%) that leverage parallel test-time compute by sampling multiple sequences and selecting the single best via an internal scoring model.
560
+ </sup><br/><sup>
561
+ • To ensure the stability of the evaluation, we employed avg@k on the AIME, HMMT, CNMO, PolyMath-en, GPQA-Diamond, EvalPlus, Tau2.
562
+ </sup><br/><sup>
563
+ • Some data points have been omitted due to prohibitively expensive evaluation costs.
564
+ </sup>
565
+
566
+ ---
567
+
568
+ #### Base model evaluation results
569
+
570
+ <div align="center">
571
+
572
+ <table>
573
+ <thead>
574
+ <tr>
575
+ <th align="center">Benchmark</th>
576
+ <th align="center">Metric</th>
577
+ <th align="center">Shot</th>
578
+ <th align="center">Kimi K2 Base</th>
579
+ <th align="center">Deepseek-V3-Base</th>
580
+ <th align="center">Qwen2.5-72B</th>
581
+ <th align="center">Llama 4 Maverick</th>
582
+ </tr>
583
+ </thead>
584
+ <tbody>
585
+ <tr>
586
+ <td align="center" colspan="7"><strong>General Tasks</strong></td>
587
+ </tr>
588
+ <tr>
589
+ <td align="center">MMLU</td>
590
+ <td align="center">EM</td>
591
+ <td align="center">5-shot</td>
592
+ <td align="center"><strong>87.8</strong></td>
593
+ <td align="center">87.1</td>
594
+ <td align="center">86.1</td>
595
+ <td align="center">84.9</td>
596
+ </tr>
597
+ <tr>
598
+ <td align="center">MMLU-pro</td>
599
+ <td align="center">EM</td>
600
+ <td align="center">5-shot</td>
601
+ <td align="center"><strong>69.2</strong></td>
602
+ <td align="center">60.6</td>
603
+ <td align="center">62.8</td>
604
+ <td align="center">63.5</td>
605
+ </tr>
606
+ <tr>
607
+ <td align="center">MMLU-redux-2.0</td>
608
+ <td align="center">EM</td>
609
+ <td align="center">5-shot</td>
610
+ <td align="center"><strong>90.2</strong></td>
611
+ <td align="center">89.5</td>
612
+ <td align="center">87.8</td>
613
+ <td align="center">88.2</td>
614
+ </tr>
615
+ <tr>
616
+ <td align="center">SimpleQA</td>
617
+ <td align="center">Correct</td>
618
+ <td align="center">5-shot</td>
619
+ <td align="center"><strong>35.3</strong></td>
620
+ <td align="center">26.5</td>
621
+ <td align="center">10.3</td>
622
+ <td align="center">23.7</td>
623
+ </tr>
624
+ <tr>
625
+ <td align="center">TriviaQA</td>
626
+ <td align="center">EM</td>
627
+ <td align="center">5-shot</td>
628
+ <td align="center"><strong>85.1</strong></td>
629
+ <td align="center">84.1</td>
630
+ <td align="center">76.0</td>
631
+ <td align="center">79.3</td>
632
+ </tr>
633
+ <tr>
634
+ <td align="center">GPQA-Diamond</td>
635
+ <td align="center">Avg@8</td>
636
+ <td align="center">5-shot</td>
637
+ <td align="center">48.1</td>
638
+ <td align="center"><strong>50.5</strong></td>
639
+ <td align="center">40.8</td>
640
+ <td align="center">49.4</td>
641
+ </tr>
642
+ <tr>
643
+ <td align="center">SuperGPQA</td>
644
+ <td align="center">EM</td>
645
+ <td align="center">5-shot</td>
646
+ <td align="center"><strong>44.7</strong></td>
647
+ <td align="center">39.2</td>
648
+ <td align="center">34.2</td>
649
+ <td align="center">38.8</td>
650
+ </tr>
651
+ <tr>
652
+ <td align="center" colspan="7"><strong>Coding Tasks</strong></td>
653
+ </tr>
654
+ <tr>
655
+ <td align="center">LiveCodeBench v6</td>
656
+ <td align="center">Pass@1</td>
657
+ <td align="center">1-shot</td>
658
+ <td align="center"><strong>26.3</strong></td>
659
+ <td align="center">22.9</td>
660
+ <td align="center">21.1</td>
661
+ <td align="center">25.1</td>
662
+ </tr>
663
+ <tr>
664
+ <td align="center">EvalPlus</td>
665
+ <td align="center">Pass@1</td>
666
+ <td align="center">-</td>
667
+ <td align="center"><strong>80.3</strong></td>
668
+ <td align="center">65.6</td>
669
+ <td align="center">66.0</td>
670
+ <td align="center">65.5</td>
671
+ </tr>
672
+ <tr>
673
+ <td align="center" colspan="7"><strong>Mathematics Tasks</strong></td>
674
+ </tr>
675
+ <tr>
676
+ <td align="center">MATH</td>
677
+ <td align="center">EM</td>
678
+ <td align="center">4-shot</td>
679
+ <td align="center"><strong>70.2</strong></td>
680
+ <td align="center">60.1</td>
681
+ <td align="center">61.0</td>
682
+ <td align="center">63.0</td>
683
+ </tr>
684
+ <tr>
685
+ <td align="center">GSM8k</td>
686
+ <td align="center">EM</td>
687
+ <td align="center">8-shot</td>
688
+ <td align="center"><strong>92.1</strong></td>
689
+ <td align="center">91.7</td>
690
+ <td align="center">90.4</td>
691
+ <td align="center">86.3</td>
692
+ </tr>
693
+ <tr>
694
+ <td align="center" colspan="7"><strong>Chinese Tasks</strong></td>
695
+ </tr>
696
+ <tr>
697
+ <td align="center">C-Eval</td>
698
+ <td align="center">EM</td>
699
+ <td align="center">5-shot</td>
700
+ <td align="center"><strong>92.5</strong></td>
701
+ <td align="center">90.0</td>
702
+ <td align="center">90.9</td>
703
+ <td align="center">80.9</td>
704
+ </tr>
705
+ <tr>
706
+ <td align="center">CSimpleQA</td>
707
+ <td align="center">Correct</td>
708
+ <td align="center">5-shot</td>
709
+ <td align="center"><strong>77.6</strong></td>
710
+ <td align="center">72.1</td>
711
+ <td align="center">50.5</td>
712
+ <td align="center">53.5</td>
713
+ </tr>
714
+ </tbody>
715
+ </table>
716
+ </div>
717
+ <sup>
718
+ • We only evaluate open-source pretrained models in this work. We report results for Qwen2.5-72B because the base checkpoint for Qwen3-235B-A22B was not open-sourced at the time of our study.
719
+ </sup><br/><sup>
720
+ • All models are evaluated using the same evaluation protocol.
721
+
722
+ </sup>
723
+
724
+
725
+ ## 4. Deployment
726
+ > [!Note]
727
+ > You can access Kimi K2's API on https://platform.moonshot.ai , we provide OpenAI/Anthropic-compatible API for you.
728
+ >
729
+ > The Anthropic-compatible API maps temperature by `real_temperature = request_temperature * 0.6` for better compatible with existing applications.
730
+
731
+ Our model checkpoints are stored in the block-fp8 format, you can find it on [Huggingface](https://huggingface.co/moonshotai/Kimi-K2-Instruct).
732
+
733
+ Currently, Kimi-K2 is recommended to run on the following inference engines:
734
+
735
+ * vLLM
736
+ * SGLang
737
+ * KTransformers
738
+ * TensorRT-LLM
739
+
740
+ Deployment examples for vLLM and SGLang can be found in the [Model Deployment Guide](docs/deploy_guidance.md).
741
+
742
+ ---
743
+
744
+ ## 5. Model Usage
745
+
746
+ ### Chat Completion
747
+
748
+ Once the local inference service is up, you can interact with it through the chat endpoint:
749
+
750
+ ```python
751
+ def simple_chat(client: OpenAI, model_name: str):
752
+ messages = [
753
+ {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
754
+ {"role": "user", "content": [{"type": "text", "text": "Please give a brief self-introduction."}]},
755
+ ]
756
+ response = client.chat.completions.create(
757
+ model=model_name,
758
+ messages=messages,
759
+ stream=False,
760
+ temperature=0.6,
761
+ max_tokens=256
762
+ )
763
+ print(response.choices[0].message.content)
764
+ ```
765
+
766
+ > [!NOTE]
767
+ > The recommended temperature for Kimi-K2-Instruct is `temperature = 0.6`.
768
+ > If no special instructions are required, the system prompt above is a good default.
769
+
770
+ ---
771
+
772
+ ### Tool Calling
773
+
774
+ Kimi-K2-Instruct has strong tool-calling capabilities.
775
+ To enable them, you need to pass the list of available tools in each request, then the model will autonomously decide when and how to invoke them.
776
+
777
+ The following example demonstrates calling a weather tool end-to-end:
778
+
779
+ ```python
780
+ # Your tool implementation
781
+ def get_weather(city: str) -> dict:
782
+ return {"weather": "Sunny"}
783
+
784
+ # Tool schema definition
785
+ tools = [{
786
+ "type": "function",
787
+ "function": {
788
+ "name": "get_weather",
789
+ "description": "Retrieve current weather information. Call this when the user asks about the weather.",
790
+ "parameters": {
791
+ "type": "object",
792
+ "required": ["city"],
793
+ "properties": {
794
+ "city": {
795
+ "type": "string",
796
+ "description": "Name of the city"
797
+ }
798
+ }
799
+ }
800
+ }
801
+ }]
802
+
803
+ # Map tool names to their implementations
804
+ tool_map = {
805
+ "get_weather": get_weather
806
+ }
807
+
808
+ def tool_call_with_client(client: OpenAI, model_name: str):
809
+ messages = [
810
+ {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
811
+ {"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
812
+ ]
813
+ finish_reason = None
814
+ while finish_reason is None or finish_reason == "tool_calls":
815
+ completion = client.chat.completions.create(
816
+ model=model_name,
817
+ messages=messages,
818
+ temperature=0.6,
819
+ tools=tools, # tool list defined above
820
+ tool_choice="auto"
821
+ )
822
+ choice = completion.choices[0]
823
+ finish_reason = choice.finish_reason
824
+ if finish_reason == "tool_calls":
825
+ messages.append(choice.message)
826
+ for tool_call in choice.message.tool_calls:
827
+ tool_call_name = tool_call.function.name
828
+ tool_call_arguments = json.loads(tool_call.function.arguments)
829
+ tool_function = tool_map[tool_call_name]
830
+ tool_result = tool_function(**tool_call_arguments)
831
+ print("tool_result:", tool_result)
832
+
833
+ messages.append({
834
+ "role": "tool",
835
+ "tool_call_id": tool_call.id,
836
+ "name": tool_call_name,
837
+ "content": json.dumps(tool_result)
838
+ })
839
+ print("-" * 100)
840
+ print(choice.message.content)
841
+ ```
842
+
843
+ The `tool_call_with_client` function implements the pipeline from user query to tool execution.
844
+ This pipeline requires the inference engine to support Kimi-K2’s native tool-parsing logic.
845
+ For streaming output and manual tool-parsing, see the [Tool Calling Guide](docs/tool_call_guidance.md).
846
+
847
+ ---
848
+
849
+ ## 6. License
850
+
851
+ Both the code repository and the model weights are released under the [Modified MIT License](LICENSE).
852
+
853
+ ---
854
+
855
+ ## 7. Contact Us
856
+
857
+ If you have any questions, please reach out at [[email protected]](mailto:[email protected]).
chat_template.jinja ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {% if tools -%}
2
+ {{ '<|im_system|>tool_declare<|im_middle|>' -}}
3
+ {{- tools | tojson -}}
4
+ {{ '<|im_end|>' -}}
5
+ {%- endif -%}
6
+
7
+ {%- for message in messages -%}
8
+ {%- if loop.first and messages[0]['role'] != 'system' -%}
9
+ {{ '<|im_system|>system<|im_middle|>You are a helpful assistant<|im_end|>' }}
10
+ {%- endif -%}
11
+ {%- if message['role'] == 'system' -%}
12
+ {{ '<|im_system|>system<|im_middle|>' }}
13
+ {%- elif message['role'] == 'user' -%}
14
+ {{ '<|im_user|>user<|im_middle|>' }}
15
+ {%- elif message['role'] == 'assistant' -%}
16
+ {{ '<|im_assistant|>assistant<|im_middle|>' }}
17
+ {%- elif message['role'] == 'tool' -%}
18
+ {{ '<|im_system|>tool<|im_middle|>' }}
19
+ {%- endif -%}
20
+
21
+ {%- if message['content'] is string -%}
22
+ {{- message['content'] + '<|im_end|>' -}}
23
+ {%- else -%}
24
+ {%- for content in message['content'] -%}
25
+ {%- if content['type'] == 'image' or 'image' in content or 'image_url' in content -%}
26
+ {{ '<|media_start|>image<|media_content|><|media_pad|><|media_end|>' }}
27
+ {%- else -%}
28
+ {{ content['text'] }}
29
+ {%- endif -%}
30
+ {%- endfor -%}
31
+ {{ '<|im_end|>' }}
32
+ {%- endif -%}
33
+ {%- endfor -%}
34
+
35
+ {%- if add_generation_prompt -%}
36
+ {{ '<|im_assistant|>assistant<|im_middle|>' }}
37
+ {%- endif -%}
config.json ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "DeepseekV3ForCausalLM"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "auto_map": {
8
+ "AutoConfig": "configuration_deepseek.DeepseekV3Config",
9
+ "AutoModel": "modeling_deepseek.DeepseekV3Model",
10
+ "AutoModelForCausalLM": "modeling_deepseek.DeepseekV3ForCausalLM"
11
+ },
12
+ "aux_loss_alpha": 0.001,
13
+ "bos_token_id": 163584,
14
+ "eos_token_id": 163585,
15
+ "ep_size": 1,
16
+ "first_k_dense_replace": 1,
17
+ "hidden_act": "silu",
18
+ "hidden_size": 7168,
19
+ "initializer_range": 0.02,
20
+ "intermediate_size": 18432,
21
+ "kv_lora_rank": 512,
22
+ "max_position_embeddings": 131072,
23
+ "model_type": "deepseek_v3",
24
+ "moe_intermediate_size": 2048,
25
+ "moe_layer_freq": 1,
26
+ "n_group": 1,
27
+ "n_routed_experts": 384,
28
+ "n_shared_experts": 1,
29
+ "norm_topk_prob": true,
30
+ "num_attention_heads": 64,
31
+ "num_experts_per_tok": 8,
32
+ "num_hidden_layers": 61,
33
+ "num_key_value_heads": 64,
34
+ "num_nextn_predict_layers": 0,
35
+ "pad_token_id": 163839,
36
+ "pretraining_tp": 1,
37
+ "q_lora_rank": 1536,
38
+ "qk_nope_head_dim": 128,
39
+ "qk_rope_head_dim": 64,
40
+ "quantization_config": {
41
+ "activation_scheme": "dynamic",
42
+ "fmt": "e4m3",
43
+ "quant_method": "fp8",
44
+ "weight_block_size": [
45
+ 128,
46
+ 128
47
+ ]
48
+ },
49
+ "rms_norm_eps": 1e-06,
50
+ "rope_scaling": {
51
+ "beta_fast": 1.0,
52
+ "beta_slow": 1.0,
53
+ "factor": 32.0,
54
+ "mscale": 1.0,
55
+ "mscale_all_dim": 1.0,
56
+ "original_max_position_embeddings": 4096,
57
+ "type": "yarn"
58
+ },
59
+ "rope_theta": 50000.0,
60
+ "routed_scaling_factor": 2.827,
61
+ "scoring_func": "sigmoid",
62
+ "seq_aux": true,
63
+ "tie_word_embeddings": false,
64
+ "topk_group": 1,
65
+ "topk_method": "noaux_tc",
66
+ "torch_dtype": "bfloat16",
67
+ "transformers_version": "4.53.2",
68
+ "unsloth_fixed": true,
69
+ "use_cache": true,
70
+ "v_head_dim": 128,
71
+ "vocab_size": 163840
72
+ }
configuration_deepseek.py ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Copy from https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/configuration_deepseek.py
2
+
3
+ from transformers.configuration_utils import PretrainedConfig
4
+ from transformers.utils import logging
5
+
6
+ logger = logging.get_logger(__name__)
7
+
8
+ DEEPSEEK_PRETRAINED_CONFIG_ARCHIVE_MAP = {}
9
+ class DeepseekV3Config(PretrainedConfig):
10
+ r"""
11
+ This is the configuration class to store the configuration of a [`DeepseekV3Model`]. It is used to instantiate an DeepSeek
12
+ model according to the specified arguments, defining the model architecture. Instantiating a configuration with the
13
+ defaults will yield a similar configuration to that of the DeepSeek-V3.
14
+
15
+ Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the
16
+ documentation from [`PretrainedConfig`] for more information.
17
+
18
+
19
+ Args:
20
+ vocab_size (`int`, *optional*, defaults to 129280):
21
+ Vocabulary size of the Deep model. Defines the number of different tokens that can be represented by the
22
+ `inputs_ids` passed when calling [`DeepseekV3Model`]
23
+ hidden_size (`int`, *optional*, defaults to 4096):
24
+ Dimension of the hidden representations.
25
+ intermediate_size (`int`, *optional*, defaults to 11008):
26
+ Dimension of the MLP representations.
27
+ moe_intermediate_size (`int`, *optional*, defaults to 1407):
28
+ Dimension of the MoE representations.
29
+ num_hidden_layers (`int`, *optional*, defaults to 32):
30
+ Number of hidden layers in the Transformer decoder.
31
+ num_nextn_predict_layers (`int`, *optional*, defaults to 1):
32
+ Number of nextn predict layers in the DeepSeekV3 Model.
33
+ num_attention_heads (`int`, *optional*, defaults to 32):
34
+ Number of attention heads for each attention layer in the Transformer decoder.
35
+ n_shared_experts (`int`, *optional*, defaults to None):
36
+ Number of shared experts, None means dense model.
37
+ n_routed_experts (`int`, *optional*, defaults to None):
38
+ Number of routed experts, None means dense model.
39
+ routed_scaling_factor (`float`, *optional*, defaults to 1.0):
40
+ Scaling factor or routed experts.
41
+ topk_method (`str`, *optional*, defaults to `gready`):
42
+ Topk method used in routed gate.
43
+ n_group (`int`, *optional*, defaults to None):
44
+ Number of groups for routed experts.
45
+ topk_group (`int`, *optional*, defaults to None):
46
+ Number of selected groups for each token(for each token, ensuring the selected experts is only within `topk_group` groups).
47
+ num_experts_per_tok (`int`, *optional*, defaults to None):
48
+ Number of selected experts, None means dense model.
49
+ moe_layer_freq (`int`, *optional*, defaults to 1):
50
+ The frequency of the MoE layer: one expert layer for every `moe_layer_freq - 1` dense layers.
51
+ first_k_dense_replace (`int`, *optional*, defaults to 0):
52
+ Number of dense layers in shallow layers(embed->dense->dense->...->dense->moe->moe...->lm_head).
53
+ \--k dense layers--/
54
+ norm_topk_prob (`bool`, *optional*, defaults to False):
55
+ Whether to normalize the weights of the routed experts.
56
+ scoring_func (`str`, *optional*, defaults to 'softmax'):
57
+ Method of computing expert weights.
58
+ aux_loss_alpha (`float`, *optional*, defaults to 0.001):
59
+ Auxiliary loss weight coefficient.
60
+ seq_aux = (`bool`, *optional*, defaults to True):
61
+ Whether to compute the auxiliary loss for each individual sample.
62
+ num_key_value_heads (`int`, *optional*):
63
+ This is the number of key_value heads that should be used to implement Grouped Query Attention. If
64
+ `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if
65
+ `num_key_value_heads=1 the model will use Multi Query Attention (MQA) otherwise GQA is used. When
66
+ converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed
67
+ by meanpooling all the original heads within that group. For more details checkout [this
68
+ paper](https://arxiv.org/pdf/2305.13245.pdf). If it is not specified, will default to
69
+ `num_attention_heads`.
70
+ hidden_act (`str` or `function`, *optional*, defaults to `"silu"`):
71
+ The non-linear activation function (function or string) in the decoder.
72
+ max_position_embeddings (`int`, *optional*, defaults to 2048):
73
+ The maximum sequence length that this model might ever be used with.
74
+ initializer_range (`float`, *optional*, defaults to 0.02):
75
+ The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
76
+ rms_norm_eps (`float`, *optional*, defaults to 1e-06):
77
+ The epsilon used by the rms normalization layers.
78
+ use_cache (`bool`, *optional*, defaults to `True`):
79
+ Whether or not the model should return the last key/values attentions (not used by all models). Only
80
+ relevant if `config.is_decoder=True`.
81
+ pad_token_id (`int`, *optional*):
82
+ Padding token id.
83
+ bos_token_id (`int`, *optional*, defaults to 1):
84
+ Beginning of stream token id.
85
+ eos_token_id (`int`, *optional*, defaults to 2):
86
+ End of stream token id.
87
+ pretraining_tp (`int`, *optional*, defaults to 1):
88
+ Experimental feature. Tensor parallelism rank used during pretraining. Please refer to [this
89
+ document](https://huggingface.co/docs/transformers/parallelism) to understand more about it. This value is
90
+ necessary to ensure exact reproducibility of the pretraining results. Please refer to [this
91
+ issue](https://github.com/pytorch/pytorch/issues/76232).
92
+ tie_word_embeddings (`bool`, *optional*, defaults to `False`):
93
+ Whether to tie weight embeddings
94
+ rope_theta (`float`, *optional*, defaults to 10000.0):
95
+ The base period of the RoPE embeddings.
96
+ rope_scaling (`Dict`, *optional*):
97
+ Dictionary containing the scaling configuration for the RoPE embeddings. Currently supports two scaling
98
+ strategies: linear and dynamic. Their scaling factor must be a float greater than 1. The expected format is
99
+ `{"type": strategy name, "factor": scaling factor}`. When using this flag, don't update
100
+ `max_position_embeddings` to the expected new maximum.
101
+ attention_bias (`bool`, defaults to `False`, *optional*, defaults to `False`):
102
+ Whether to use a bias in the query, key, value and output projection layers during self-attention.
103
+ attention_dropout (`float`, *optional*, defaults to 0.0):
104
+ The dropout ratio for the attention probabilities.
105
+
106
+ ```python
107
+ >>> from transformers import DeepseekV3Model, DeepseekV3Config
108
+
109
+ >>> # Initializing a Deepseek-V3 style configuration
110
+ >>> configuration = DeepseekV3Config()
111
+
112
+ >>> # Accessing the model configuration
113
+ >>> configuration = model.config
114
+ ```"""
115
+
116
+ model_type = "deepseek_v3"
117
+ keys_to_ignore_at_inference = ["past_key_values"]
118
+
119
+ def __init__(
120
+ self,
121
+ vocab_size=129280,
122
+ hidden_size=7168,
123
+ intermediate_size=18432,
124
+ moe_intermediate_size = 2048,
125
+ num_hidden_layers=61,
126
+ num_nextn_predict_layers=1,
127
+ num_attention_heads=128,
128
+ num_key_value_heads=128,
129
+ n_shared_experts = 1,
130
+ n_routed_experts = 256,
131
+ ep_size = 1,
132
+ routed_scaling_factor = 2.5,
133
+ kv_lora_rank = 512,
134
+ q_lora_rank = 1536,
135
+ qk_rope_head_dim = 64,
136
+ v_head_dim = 128,
137
+ qk_nope_head_dim = 128,
138
+ topk_method = 'noaux_tc',
139
+ n_group = 8,
140
+ topk_group = 4,
141
+ num_experts_per_tok = 8,
142
+ moe_layer_freq = 1,
143
+ first_k_dense_replace = 3,
144
+ norm_topk_prob = True,
145
+ scoring_func = 'sigmoid',
146
+ aux_loss_alpha = 0.001,
147
+ seq_aux = True,
148
+ hidden_act="silu",
149
+ max_position_embeddings=4096,
150
+ initializer_range=0.02,
151
+ rms_norm_eps=1e-6,
152
+ use_cache=True,
153
+ pad_token_id=None,
154
+ bos_token_id=0,
155
+ eos_token_id=1,
156
+ pretraining_tp=1,
157
+ tie_word_embeddings=False,
158
+ rope_theta=10000.0,
159
+ rope_scaling=None,
160
+ attention_bias=False,
161
+ attention_dropout=0.0,
162
+ **kwargs,
163
+ ):
164
+ self.vocab_size = vocab_size
165
+ self.max_position_embeddings = max_position_embeddings
166
+ self.hidden_size = hidden_size
167
+ self.intermediate_size = intermediate_size
168
+ self.moe_intermediate_size = moe_intermediate_size
169
+ self.num_hidden_layers = num_hidden_layers
170
+ self.num_nextn_predict_layers = num_nextn_predict_layers
171
+ self.num_attention_heads = num_attention_heads
172
+ self.n_shared_experts = n_shared_experts
173
+ self.n_routed_experts = n_routed_experts
174
+ self.ep_size = ep_size
175
+ self.routed_scaling_factor = routed_scaling_factor
176
+ self.kv_lora_rank = kv_lora_rank
177
+ self.q_lora_rank = q_lora_rank
178
+ self.qk_rope_head_dim = qk_rope_head_dim
179
+ self.v_head_dim = v_head_dim
180
+ self.qk_nope_head_dim = qk_nope_head_dim
181
+ self.topk_method = topk_method
182
+ self.n_group = n_group
183
+ self.topk_group = topk_group
184
+ self.num_experts_per_tok = num_experts_per_tok
185
+ self.moe_layer_freq = moe_layer_freq
186
+ self.first_k_dense_replace = first_k_dense_replace
187
+ self.norm_topk_prob = norm_topk_prob
188
+ self.scoring_func = scoring_func
189
+ self.aux_loss_alpha = aux_loss_alpha
190
+ self.seq_aux = seq_aux
191
+ # for backward compatibility
192
+ if num_key_value_heads is None:
193
+ num_key_value_heads = num_attention_heads
194
+
195
+ self.num_key_value_heads = num_key_value_heads
196
+ self.hidden_act = hidden_act
197
+ self.initializer_range = initializer_range
198
+ self.rms_norm_eps = rms_norm_eps
199
+ self.pretraining_tp = pretraining_tp
200
+ self.use_cache = use_cache
201
+ self.rope_theta = rope_theta
202
+ self.rope_scaling = rope_scaling
203
+ self.attention_bias = attention_bias
204
+ self.attention_dropout = attention_dropout
205
+
206
+ super().__init__(
207
+ pad_token_id=pad_token_id,
208
+ bos_token_id=bos_token_id,
209
+ eos_token_id=eos_token_id,
210
+ tie_word_embeddings=tie_word_embeddings,
211
+ **kwargs,
212
+ )
generation_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_length": 131072,
3
+ "eos_token_id": 163586
4
+ }
kimi-logo.png ADDED
model-1-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7383ae82886098e3355c7f58c806acc23b2eb021e7fad6fa406b6528841425b7
3
+ size 3343812312
model-10-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b21d08f2cb6dc040e256ee9871c6aa151bbba09f51a017ae9fc11d1b55a15e19
3
+ size 34118878784
model-11-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb2220d6d97dd218acc9a3f22d275cbce99465e436abbeb760372483917f6ae7
3
+ size 34118879952
model-12-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:33e0a357983e1ac3765ec8bf07928a6764e2540116be00bc8ffa95a0bbbeec6b
3
+ size 34118879952
model-13-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:83141a6c84028a4b4f65c162c086a97b09dc4e019b4c5e521fa607432dcaa0c4
3
+ size 34118879952
model-14-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aba47f6864348191641d1187fe4dcabd0197180254a51e02797825bcd7a096c0
3
+ size 34118879952
model-15-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:565ddc751445902ba413eb566e03118224f0905432659f0ffe20416ab65ebdd0
3
+ size 34118879952
model-16-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:47fadafbc0be9dd9b08709f0509a5e782d96a06e829a9b97522a93d25dbc142b
3
+ size 34118879952
model-17-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:427226868a21cb6a4704a5a744ff08c9a3a75f531e1eeb92a02c56767fe91d52
3
+ size 34118879952
model-18-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0bbc10edf4cf6bb603cb284f251e9193b498b962fcff8e941661a835b29febbf
3
+ size 34118879952
model-19-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8cf771ece90219517290dc8d3bde1a2db68343bd376d64d4ef890cf0a2d6fe3d
3
+ size 34118879952
model-2-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de7f6006c802b0da60dddcc01029b67a9b9467bc4890401a81344f0a9fb9d51a
3
+ size 34118878784
model-20-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:572dfc07d04237b356af6e118c333fe3dbfc25f9f21f01f35a7651f711a8a336
3
+ size 34118879952
model-21-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2efc6151e5bb8dbd5612995b9c2ed3cdb3dc8711b24398e4d6a51f1791f0793f
3
+ size 34118879952
model-22-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ebf12f7c32d3793deebc83384fc1ffea2a9e191467ec67b4048d4dd9cecae3ae
3
+ size 34118879952
model-23-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aaa9a31a77789432354d7cec5265183a28b7e77b8a0bf0ab5c6e42ef72696484
3
+ size 34118879952
model-24-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51ae5e19d8832296832b19d3c0d4d8c542b24804ec3285b5df4e26fb24313eb9
3
+ size 34118879952
model-25-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fcf4477fa1d57e795c063fb0e5c64790784efc4b56e62fe18a263ab3c218f24c
3
+ size 34118879952
model-26-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5db68fe408713507643078c055b1f1611ed8fb7d0d73f83a09a283ff29d8d076
3
+ size 34118879952
model-27-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7baa5bd26201325d4831c965b1b4c915665d3aea99ecacf74e4d66758369f80e
3
+ size 34118879952
model-28-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a5b3d405c6bbaf5b54df594fb30391f17df94c4ef499c674e628d31f72f61d56
3
+ size 34118879952
model-29-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6b27b9eb8b9ace62a1eaaac0960ead5f900781da88bdd9cbf42ab78aae5106d5
3
+ size 34118879952
model-3-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ff67e39323325486b3cd7ddf7544e693cb2dbdfb052863c7b6248163a10230f2
3
+ size 34118878784
model-30-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5166f475245b4175d74979802fa21ce264e8f5d42bb27034597133c362c28307
3
+ size 34118879952
model-31-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71215a48d38a45cdcc173ed78a6fa730546cd1a78e1cf8bd0f4017fd9f643511
3
+ size 34118879952
model-32-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba1b1db54c6d0d559b58f69aa472998ae54a876f2271e300ed155ca1eff13ce0
3
+ size 34118879952
model-33-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c48bfa444ea8ef9bea51c270011cbafb24074a2cc3ea784135ea3c4dac87152
3
+ size 34118879952
model-34-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9abf313b1b0ee85970af16d07bbf075b938a18b75de67935d53abad408db9181
3
+ size 34118879952
model-35-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e390859e25b0b9db65a9d0501ae3d2690a3d62a1a75c18978c8f13e4fb46640c
3
+ size 34118879952
model-36-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:947ab77bdc30cee2b195bb130eed6078b4a0f5b5abcd147e1599daf718eb7c91
3
+ size 34118879952
model-37-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c2bdaa00a5298742c328e7754d553dcf3ada5702dc2bf73cfb09806a167b5b2
3
+ size 34118879952
model-38-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b90541bdb09e994cb747c22c0ed35ba143f34cb4c64089e0fead2a4b0863322e
3
+ size 34118879952
model-39-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba6b1e32cb1ebefb97d5c884b6f06d646300c21f05bbf6c12e3e866faf1b20ee
3
+ size 34118879952
model-4-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d5d4880793242de89a4e23f494295642d99184d7f58fd3c9c962ccdcffd2121b
3
+ size 34118878784
model-40-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86fe2f537295c6ff4f6c4c58a9679dee694d50eff54dc5aea222bb8f6a68ff8a
3
+ size 34118879952
model-41-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:643d53ed16f7b214af94d66d5a0b942490d36b93d3e1c70ec0a11105838e1206
3
+ size 34118879952
model-42-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8798cec89754200186e06b5c6406cdb7efe48b1aefd52df90708b46754435337
3
+ size 34118879952
model-43-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:86f656e23695230bf44124e5ffea130ac45ca011eac29a8f1e86c7c509feeff6
3
+ size 34118879952
model-44-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc95a4d5305de7b7e6fe0a20a7da7dfded22fe4a1e17280078ef1278703588d4
3
+ size 34118879952
model-45-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e89ab6450c8720244ccf561cbbe1b36674c25457a77f249ca3a02fd7f4f57e1e
3
+ size 34118879952
model-46-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bc40a7c696b21d5d66d2f4a9eb7ff761d1cd09919c21cab36343266d4f8b268f
3
+ size 34118879952
model-47-of-61.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:de069f96482c863e2d5356cc6a2377452e7a4cabdba1238b99749feadc5f41ff
3
+ size 34118879952