ivanfioravanti commited on
Commit
a70db81
·
verified ·
1 Parent(s): afcba2c

Add files using upload-large-folder tool

Browse files
Files changed (2) hide show
  1. README.md +28 -0
  2. config.json +0 -0
README.md CHANGED
@@ -6,3 +6,31 @@ tags:
6
  - mlx
7
  pipeline_tag: text-generation
8
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  - mlx
7
  pipeline_tag: text-generation
8
  ---
9
+
10
+ # mlx-community/DeepSeek-R1-0528-5bit
11
+
12
+ This model [mlx-community/DeepSeek-R1-0528-5bit](https://huggingface.co/mlx-community/DeepSeek-R1-0528-5bit) was
13
+ converted to MLX format from [deepseek-ai/DeepSeek-R1-0528](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)
14
+ using mlx-lm version **0.25.3**.
15
+
16
+ ## Use with mlx
17
+
18
+ ```bash
19
+ pip install mlx-lm
20
+ ```
21
+
22
+ ```python
23
+ from mlx_lm import load, generate
24
+
25
+ model, tokenizer = load("mlx-community/DeepSeek-R1-0528-5bit")
26
+
27
+ prompt = "hello"
28
+
29
+ if tokenizer.chat_template is not None:
30
+ messages = [{"role": "user", "content": prompt}]
31
+ prompt = tokenizer.apply_chat_template(
32
+ messages, add_generation_prompt=True
33
+ )
34
+
35
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
36
+ ```
config.json CHANGED
The diff for this file is too large to render. See raw diff