Hugging Face's logo

ibm-granite
/
granite-3.3-2b-instruct

Text Generation
Transformers
Safetensors
granite
language
granite-3.3
conversational
Model card Files
xet
Community
5
ranarag commited on
Commit
1f4481a
·
verified ·
1 Parent(s): de39e7d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +104 -115
README.md CHANGED
@@ -207,10 +207,10 @@ By implementing this innovative prevention strategy, we can significantly reduce
207
  **Evaluation Results:**
208
  <table>
209
  <thead>
210
- <caption style="text-align:center"><b>Comparison with different models over various benchmarks. Scores of AlpacaEval-2.0 and Arena-Hard are calculated with thinking=True</b><sup id="fnref1"><a href="#fn1">1</a></caption>
211
  <tr>
212
  <th style="text-align:left; background-color: #001d6c; color: white;">Models</th>
213
- <th style="text-align:center; background-color: #001d6c; color: white;">ArenaHard</th>
214
  <th style="text-align:center; background-color: #001d6c; color: white;">AlpacaEval-2.0</th>
215
  <th style="text-align:center; background-color: #001d6c; color: white;">MMLU</th>
216
  <th style="text-align:center; background-color: #001d6c; color: white;">PopQA</th>
@@ -255,114 +255,114 @@ By implementing this innovative prevention strategy, we can significantly reduce
255
  <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">83.23</td>
256
  </tr>
257
  <tr>
258
- <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"><b>Granite-3.3-2B-Instruct</b></td>
259
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 28.86 </td>
260
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 43.45 </td>
261
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 55.88 </td>
262
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 18.4 </td>
263
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 58.97 </td>
264
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 52.51 </td>
265
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 35.98 </td>
266
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 72.48 </td>
267
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 80.51 </td>
268
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 75.68 </td>
269
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 65.8 </td>
270
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">87.47</td>
271
  </tr>
272
 
273
  <tr>
274
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Llama-3.1-8B-Instruct</td>
275
- <td style="text-align:center; background-color: #DAE8FF; color: black;">36.43</td>
276
- <td style="text-align:center; background-color: #DAE8FF; color: black;">27.22</td>
277
- <td style="text-align:center; background-color: #DAE8FF; color: black;">69.15</td>
278
- <td style="text-align:center; background-color: #DAE8FF; color: black;">28.79</td>
279
- <td style="text-align:center; background-color: #DAE8FF; color: black;">52.79</td>
280
- <td style="text-align:center; background-color: #DAE8FF; color: black;">72.66</td>
281
- <td style="text-align:center; background-color: #DAE8FF; color: black;">61.48</td>
282
- <td style="text-align:center; background-color: #DAE8FF; color: black;">83.24</td>
283
- <td style="text-align:center; background-color: #DAE8FF; color: black;">85.32</td>
284
- <td style="text-align:center; background-color: #DAE8FF; color: black;">80.15</td>
285
- <td style="text-align:center; background-color: #DAE8FF; color: black;">79.10</td>
286
- <td style="text-align:center; background-color: #DAE8FF; color: black;">83.43</td>
287
  </tr>
288
 
289
  <tr>
290
- <td style="text-align:left; background-color: #DAE8FF; color: black;">DeepSeek-R1-Distill-Llama-8B</td>
291
- <td style="text-align:center; background-color: #DAE8FF; color: black;">17.17</td>
292
- <td style="text-align:center; background-color: #DAE8FF; color: black;">21.85</td>
293
- <td style="text-align:center; background-color: #DAE8FF; color: black;">45.80</td>
294
- <td style="text-align:center; background-color: #DAE8FF; color: black;">13.25</td>
295
- <td style="text-align:center; background-color: #DAE8FF; color: black;">47.43</td>
296
- <td style="text-align:center; background-color: #DAE8FF; color: black;">65.71</td>
297
- <td style="text-align:center; background-color: #DAE8FF; color: black;">44.46</td>
298
- <td style="text-align:center; background-color: #DAE8FF; color: black;">72.18</td>
299
- <td style="text-align:center; background-color: #DAE8FF; color: black;">67.54</td>
300
- <td style="text-align:center; background-color: #DAE8FF; color: black;">62.91</td>
301
- <td style="text-align:center; background-color: #DAE8FF; color: black;">66.50</td>
302
- <td style="text-align:center; background-color: #DAE8FF; color: black;">42.87</td>
303
  </tr>
304
 
305
  <tr>
306
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Qwen-2.5-7B-Instruct</td>
307
- <td style="text-align:center; background-color: #DAE8FF; color: black;">25.44</td>
308
- <td style="text-align:center; background-color: #DAE8FF; color: black;">30.34</td>
309
- <td style="text-align:center; background-color: #DAE8FF; color: black;">74.30</td>
310
- <td style="text-align:center; background-color: #DAE8FF; color: black;">18.12</td>
311
- <td style="text-align:center; background-color: #DAE8FF; color: black;">63.06</td>
312
- <td style="text-align:center; background-color: #DAE8FF; color: black;">70.40</td>
313
- <td style="text-align:center; background-color: #DAE8FF; color: black;">54.71</td>
314
- <td style="text-align:center; background-color: #DAE8FF; color: black;">84.46</td>
315
- <td style="text-align:center; background-color: #DAE8FF; color: black;">93.35</td>
316
- <td style="text-align:center; background-color: #DAE8FF; color: black;">89.91</td>
317
- <td style="text-align:center; background-color: #DAE8FF; color: black;">74.90</td>
318
- <td style="text-align:center; background-color: #DAE8FF; color: black;">81.90</td>
319
  </tr>
320
 
321
  <tr>
322
- <td style="text-align:left; background-color: #DAE8FF; color: black;">DeepSeek-R1-Distill-Qwen-7B</td>
323
- <td style="text-align:center; background-color: #DAE8FF; color: black;">10.36</td>
324
- <td style="text-align:center; background-color: #DAE8FF; color: black;">15.35</td>
325
- <td style="text-align:center; background-color: #DAE8FF; color: black;">50.72</td>
326
- <td style="text-align:center; background-color: #DAE8FF; color: black;">9.94</td>
327
- <td style="text-align:center; background-color: #DAE8FF; color: black;">47.14</td>
328
- <td style="text-align:center; background-color: #DAE8FF; color: black;">65.04</td>
329
- <td style="text-align:center; background-color: #DAE8FF; color: black;">42.76</td>
330
- <td style="text-align:center; background-color: #DAE8FF; color: black;">78.47</td>
331
- <td style="text-align:center; background-color: #DAE8FF; color: black;">79.89</td>
332
- <td style="text-align:center; background-color: #DAE8FF; color: black;">78.43</td>
333
- <td style="text-align:center; background-color: #DAE8FF; color: black;">59.10</td>
334
- <td style="text-align:center; background-color: #DAE8FF; color: black;">42.45</td>
335
  </tr>
336
  <tr>
337
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Granite-3.1-8B-Instruct</td>
338
- <td style="text-align:center; background-color: #DAE8FF; color: black;">37.58</td>
339
- <td style="text-align:center; background-color: #DAE8FF; color: black;">30.34</td>
340
- <td style="text-align:center; background-color: #DAE8FF; color: black;">66.77</td>
341
- <td style="text-align:center; background-color: #DAE8FF; color: black;">28.7</td>
342
- <td style="text-align:center; background-color: #DAE8FF; color: black;">65.84</td>
343
- <td style="text-align:center; background-color: #DAE8FF; color: black;">68.55</td>
344
- <td style="text-align:center; background-color: #DAE8FF; color: black;">50.78</td>
345
- <td style="text-align:center; background-color: #DAE8FF; color: black;">79.15</td>
346
- <td style="text-align:center; background-color: #DAE8FF; color: black;">89.63</td>
347
- <td style="text-align:center; background-color: #DAE8FF; color: black;">85.79</td>
348
- <td style="text-align:center; background-color: #DAE8FF; color: black;">73.20</td>
349
- <td style="text-align:center; background-color: #DAE8FF; color: black;">85.73</td>
350
  </tr>
351
 
352
  <tr>
353
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Granite-3.2-8B-Instruct</td>
354
- <td style="text-align:center; background-color: #DAE8FF; color: black;">55.25</td>
355
- <td style="text-align:center; background-color: #DAE8FF; color: black;">61.19</td>
356
- <td style="text-align:center; background-color: #DAE8FF; color: black;">66.79</td>
357
- <td style="text-align:center; background-color: #DAE8FF; color: black;">28.04</td>
358
- <td style="text-align:center; background-color: #DAE8FF; color: black;">66.92</td>
359
- <td style="text-align:center; background-color: #DAE8FF; color: black;">64.77</td>
360
- <td style="text-align:center; background-color: #DAE8FF; color: black;">50.95</td>
361
- <td style="text-align:center; background-color: #DAE8FF; color: black;">81.65</td>
362
- <td style="text-align:center; background-color: #DAE8FF; color: black;">89.35</td>
363
- <td style="text-align:center; background-color: #DAE8FF; color: black;">85.72</td>
364
- <td style="text-align:center; background-color: #DAE8FF; color: black;">74.31</td>
365
- <td style="text-align:center; background-color: #DAE8FF; color: black;">84.7</td>
366
  </tr>
367
  <tr>
368
  <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-8B-Instruct</b></td>
@@ -387,7 +387,7 @@ By implementing this innovative prevention strategy, we can significantly reduce
387
  <tr>
388
  <th style="text-align:left; background-color: #001d6c; color: white;">Models</th>
389
  <th style="text-align:center; background-color: #001d6c; color: white;">AIME24</th>
390
- <th style="text-align:center; background-color: #001d6c; color: white;">MATH500</th>
391
  </tr></thead>
392
  <tbody>
393
  <tr>
@@ -401,19 +401,19 @@ By implementing this innovative prevention strategy, we can significantly reduce
401
  <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 35.54 </td>
402
  </tr>
403
  <tr>
404
- <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;"><b>Granite-3.3-2B-Instruct</b></td>
405
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 3.28 </td>
406
- <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 58.09 </td>
407
  </tr>
408
  <tr>
409
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Granite-3.1-8B-Instruct</td>
410
- <td style="text-align:center; background-color: #DAE8FF; color: black;"> 1.97 </td>
411
- <td style="text-align:center; background-color: #DAE8FF; color: black;"> 48.73 </td>
412
  </tr>
413
  <tr>
414
- <td style="text-align:left; background-color: #DAE8FF; color: black;">Granite-3.2-8B-Instruct</td>
415
- <td style="text-align:center; background-color: #DAE8FF; color: black;"> 2.43 </td>
416
- <td style="text-align:center; background-color: #DAE8FF; color: black;"> 52.8 </td>
417
  </tr>
418
  <tr>
419
  <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-8B-Instruct</b></td>
@@ -527,15 +527,4 @@ Granite-3.3-2B-Instruct builds upon Granite-3.3-2B-Base, leveraging both permiss
527
 
528
 
529
 
530
- <p><a href="#fnref1" title="Jump back to reference">[1]</a> Evaluated using <a href="https://github.com/allenai/olmes">OLMES</a> (except the AttaQ scores)</p>
531
- <!-- ## Citation
532
- ```
533
- @misc{granite-models,
534
- author = {author 1, author2, ...},
535
- title = {},
536
- journal = {},
537
- volume = {},
538
- year = {2024},
539
- url = {https://arxiv.org/abs/0000.00000},
540
- }
541
- ``` -->
 
207
  **Evaluation Results:**
208
  <table>
209
  <thead>
210
+ <caption style="text-align:center"><b>Comparison with different models over various benchmarks<sup id="fnref1"><a href="#fn1">1</a></sup>. Scores of AlpacaEval-2.0 and Arena-Hard are calculated with thinking=True</b></caption>
211
  <tr>
212
  <th style="text-align:left; background-color: #001d6c; color: white;">Models</th>
213
+ <th style="text-align:center; background-color: #001d6c; color: white;">Arena-Hard</th>
214
  <th style="text-align:center; background-color: #001d6c; color: white;">AlpacaEval-2.0</th>
215
  <th style="text-align:center; background-color: #001d6c; color: white;">MMLU</th>
216
  <th style="text-align:center; background-color: #001d6c; color: white;">PopQA</th>
 
255
  <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">83.23</td>
256
  </tr>
257
  <tr>
258
+ <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-2B-Instruct</b></td>
259
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 28.86 </td>
260
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 43.45 </td>
261
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 55.88 </td>
262
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 18.4 </td>
263
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 58.97 </td>
264
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 52.51 </td>
265
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 35.98 </td>
266
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 72.48 </td>
267
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 80.51 </td>
268
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 75.68 </td>
269
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 65.8 </td>
270
+ <td style="text-align:center; background-color: #DAE8FF; color: black;">87.47</td>
271
  </tr>
272
 
273
  <tr>
274
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Llama-3.1-8B-Instruct</td>
275
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">36.43</td>
276
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">27.22</td>
277
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">69.15</td>
278
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">28.79</td>
279
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">52.79</td>
280
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">72.66</td>
281
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">61.48</td>
282
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">83.24</td>
283
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">85.32</td>
284
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">80.15</td>
285
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">79.10</td>
286
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">83.43</td>
287
  </tr>
288
 
289
  <tr>
290
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">DeepSeek-R1-Distill-Llama-8B</td>
291
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">17.17</td>
292
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">21.85</td>
293
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">45.80</td>
294
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">13.25</td>
295
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">47.43</td>
296
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">65.71</td>
297
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">44.46</td>
298
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">72.18</td>
299
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">67.54</td>
300
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">62.91</td>
301
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">66.50</td>
302
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">42.87</td>
303
  </tr>
304
 
305
  <tr>
306
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Qwen-2.5-7B-Instruct</td>
307
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">25.44</td>
308
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">30.34</td>
309
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">74.30</td>
310
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">18.12</td>
311
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">63.06</td>
312
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">70.40</td>
313
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">54.71</td>
314
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">84.46</td>
315
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">93.35</td>
316
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">89.91</td>
317
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">74.90</td>
318
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">81.90</td>
319
  </tr>
320
 
321
  <tr>
322
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">DeepSeek-R1-Distill-Qwen-7B</td>
323
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">10.36</td>
324
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">15.35</td>
325
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">50.72</td>
326
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">9.94</td>
327
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">47.14</td>
328
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">65.04</td>
329
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">42.76</td>
330
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">78.47</td>
331
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">79.89</td>
332
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">78.43</td>
333
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">59.10</td>
334
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">42.45</td>
335
  </tr>
336
  <tr>
337
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Granite-3.1-8B-Instruct</td>
338
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">37.58</td>
339
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">30.34</td>
340
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">66.77</td>
341
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">28.7</td>
342
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">65.84</td>
343
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">68.55</td>
344
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">50.78</td>
345
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">79.15</td>
346
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">89.63</td>
347
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">85.79</td>
348
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">73.20</td>
349
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">85.73</td>
350
  </tr>
351
 
352
  <tr>
353
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Granite-3.2-8B-Instruct</td>
354
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">55.25</td>
355
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">61.19</td>
356
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">66.79</td>
357
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">28.04</td>
358
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">66.92</td>
359
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">64.77</td>
360
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">50.95</td>
361
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">81.65</td>
362
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">89.35</td>
363
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">85.72</td>
364
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">74.31</td>
365
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;">84.7</td>
366
  </tr>
367
  <tr>
368
  <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-8B-Instruct</b></td>
 
387
  <tr>
388
  <th style="text-align:left; background-color: #001d6c; color: white;">Models</th>
389
  <th style="text-align:center; background-color: #001d6c; color: white;">AIME24</th>
390
+ <th style="text-align:center; background-color: #001d6c; color: white;">MATH-500</th>
391
  </tr></thead>
392
  <tbody>
393
  <tr>
 
401
  <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 35.54 </td>
402
  </tr>
403
  <tr>
404
+ <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-2B-Instruct</b></td>
405
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 3.28 </td>
406
+ <td style="text-align:center; background-color: #DAE8FF; color: black;"> 58.09 </td>
407
  </tr>
408
  <tr>
409
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Granite-3.1-8B-Instruct</td>
410
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 1.97 </td>
411
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 48.73 </td>
412
  </tr>
413
  <tr>
414
+ <td style="text-align:left; background-color: #FFFFFF; color: #2D2D2D;">Granite-3.2-8B-Instruct</td>
415
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 2.43 </td>
416
+ <td style="text-align:center; background-color: #FFFFFF; color: #2D2D2D;"> 52.8 </td>
417
  </tr>
418
  <tr>
419
  <td style="text-align:left; background-color: #DAE8FF; color: black;"><b>Granite-3.3-8B-Instruct</b></td>
 
527
 
528
 
529
 
530
+ <p><a href="#fnref1" title="Jump back to reference">[1]</a> Evaluated using <a href="https://github.com/allenai/olmes">OLMES</a> (except AttaQ and Arena-Hard scores)</p>