bowenbaoamd commited on
Commit
9f4609e
·
verified ·
1 Parent(s): 1cbd6dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -97,6 +97,16 @@ The model was evaluated on AIME24, GPQA Diamond, and MATH-500 benchmarks using t
97
  <td>99.84%
98
  </td>
99
  </tr>
 
 
 
 
 
 
 
 
 
 
100
  </table>
101
 
102
 
@@ -115,6 +125,18 @@ lighteval vllm $MODEL_ARGS "custom|aime24_single|0|0,custom|math_500_single|0|0,
115
  2>&1 | tee -a "$LOG"
116
  ```
117
 
 
 
 
 
 
 
 
 
 
 
 
 
118
 
119
  # License
120
  Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.
 
97
  <td>99.84%
98
  </td>
99
  </tr>
100
+ <tr>
101
+ <td>gsm8k
102
+ </td>
103
+ <td>95.30
104
+ </td>
105
+ <td>95.18
106
+ </td>
107
+ <td>99.87%
108
+ </td>
109
+ </tr>
110
  </table>
111
 
112
 
 
125
  2>&1 | tee -a "$LOG"
126
  ```
127
 
128
+ gsm8k results were obtained using lm-eval-harness, with 10 rounds using different random seeds.
129
+
130
+ ```
131
+ MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"
132
+ lm_eval \
133
+ --model local-completions \
134
+ --model_args $MODEL_ARGS \
135
+ --tasks gsm8k \
136
+ --num_fewshot 8 \
137
+ --batch_size auto
138
+ ```
139
+
140
 
141
  # License
142
  Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.