Update README.md
Browse files
README.md
CHANGED
|
@@ -97,6 +97,16 @@ The model was evaluated on AIME24, GPQA Diamond, and MATH-500 benchmarks using t
|
|
| 97 |
<td>99.84%
|
| 98 |
</td>
|
| 99 |
</tr>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
</table>
|
| 101 |
|
| 102 |
|
|
@@ -115,6 +125,18 @@ lighteval vllm $MODEL_ARGS "custom|aime24_single|0|0,custom|math_500_single|0|0,
|
|
| 115 |
2>&1 | tee -a "$LOG"
|
| 116 |
```
|
| 117 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 118 |
|
| 119 |
# License
|
| 120 |
Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.
|
|
|
|
| 97 |
<td>99.84%
|
| 98 |
</td>
|
| 99 |
</tr>
|
| 100 |
+
<tr>
|
| 101 |
+
<td>gsm8k
|
| 102 |
+
</td>
|
| 103 |
+
<td>95.30
|
| 104 |
+
</td>
|
| 105 |
+
<td>95.18
|
| 106 |
+
</td>
|
| 107 |
+
<td>99.87%
|
| 108 |
+
</td>
|
| 109 |
+
</tr>
|
| 110 |
</table>
|
| 111 |
|
| 112 |
|
|
|
|
| 125 |
2>&1 | tee -a "$LOG"
|
| 126 |
```
|
| 127 |
|
| 128 |
+
gsm8k results were obtained using lm-eval-harness, with 10 rounds using different random seeds.
|
| 129 |
+
|
| 130 |
+
```
|
| 131 |
+
MODEL_ARGS="model=amd/DeepSeek-R1-0528-MXFP4-ASQ,base_url=http://localhost:8000/v1/completions,num_concurrent=999999,timeout=999999,tokenized_requests=False,max_length=38768,temperature=0.6,top_p=0.95,add_bos_token=True,seed=$SEED"
|
| 132 |
+
lm_eval \
|
| 133 |
+
--model local-completions \
|
| 134 |
+
--model_args $MODEL_ARGS \
|
| 135 |
+
--tasks gsm8k \
|
| 136 |
+
--num_fewshot 8 \
|
| 137 |
+
--batch_size auto
|
| 138 |
+
```
|
| 139 |
+
|
| 140 |
|
| 141 |
# License
|
| 142 |
Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.
|