---
datasets:
- Marcus2112/minipile_density-proportioned
language:
- en
base_model:
- EleutherAI/pythia-160m-deduped
---

| Benchmark        | Measure    |     | 160M Density                 | 160M Density 2 Epochs          | Percentage Difference in Means |
| ---------------- | ---------- | --- | ---------------------------- | ------------------------------ | ------------------------------ |
| ARC-Challenge    | acc        | ↑   | **0.1920 ± 0.0115**          | 0.1894 ± 0.0115                | -1.3542                        |
| MMLU             | acc        | ↑   | 0.2295 ± 0.0035              | 0.2295 ± 0.0035                | 0.0000                         |
| HellaSwag        | acc        | ↑   | **0.2604 ± 0.0044**          | 0.2568 ± 0.0044                | -1.3825                        |
| WinoGrande       | acc        | ↑   | **0.5201 ± 0.0140**          | 0.5012 ± 0.0141                | -3.6339                        |
| Lambada (OpenAI) | acc        | ↑   | 0.0000 ± 0.0000              | 0.0000 ± 0.0000                | -                              |
| Lambada (OpenAI) | perplexity | ↓   | 2099002.0912 ± 170652.6222   | **1587737.3755 ± 121555.3148** | -24.3575                       |
| Lambada (Std)    | acc        | ↑   | 0.0000 ± 0.0000              | 0.0000 ± 0.0000                | -                              |
| Lambada (Std)    | perplexity | ↓   | 13347273.6076 ± 1997894.6360 | **8366924.7603 ± 713077.3579** | -37.3136                       |
| BLiMP            | acc        | ↑   | **0.5501 ± 0.0017**          | 0.5378 ± 0.0017                | -2.2360                        |