PolyPythias
- Preview • Updated • 410
EleutherAI/pile-preshuffled-seeds
Updated • 504 • 1Note Training data information for each seed.
EleutherAI/pythia-14m
Text Generation • 0.0B • Updated • 222k • 23EleutherAI/pythia-14m-seed1
Updated • 89EleutherAI/pythia-14m-seed2
Updated • 81EleutherAI/pythia-14m-seed3
Updated • 79EleutherAI/pythia-14m-seed4
Updated • 81EleutherAI/pythia-14m-seed5
Updated • 79EleutherAI/pythia-14m-seed6
Updated • 73EleutherAI/pythia-14m-seed7
Updated • 77EleutherAI/pythia-14m-seed8
Updated • 73EleutherAI/pythia-14m-seed9
Updated • 73EleutherAI/pythia-31m
Text Generation • 0.1B • Updated • 125k • 5EleutherAI/pythia-31m-seed1
Updated • 47EleutherAI/pythia-31m-seed2
Updated • 46EleutherAI/pythia-31m-seed3
Updated • 47EleutherAI/pythia-31m-seed4
Updated • 47EleutherAI/pythia-31m-seed5
Updated • 47EleutherAI/pythia-31m-seed6
Updated • 41EleutherAI/pythia-31m-seed7
Updated • 41EleutherAI/pythia-31m-seed8
Updated • 41EleutherAI/pythia-31m-seed9
Updated • 41EleutherAI/pythia-70m
0.1B • Updated • 158k • 71EleutherAI/pythia-70m-seed1
Updated • 572EleutherAI/pythia-70m-seed2
Updated • 566EleutherAI/pythia-70m-seed3
Updated • 557EleutherAI/pythia-70m-seed4
Updated • 554EleutherAI/pythia-70m-seed5
Updated • 558EleutherAI/pythia-70m-seed6
Updated • 104EleutherAI/pythia-70m-seed7
Updated • 105EleutherAI/pythia-70m-seed8
Updated • 104EleutherAI/pythia-70m-seed9
Updated • 105EleutherAI/pythia-160m
Text Generation • 0.2B • Updated • 118k • 32EleutherAI/pythia-160m-seed1
Text Generation • 0.2B • Updated • 2.34kEleutherAI/pythia-160m-seed2
Text Generation • 0.2B • Updated • 2.33kEleutherAI/pythia-160m-seed3
Text Generation • 0.2B • Updated • 2.33kEleutherAI/pythia-160m-seed4
Updated • 110 • 1EleutherAI/pythia-160m-seed5
Updated • 109EleutherAI/pythia-160m-seed6
Updated • 103EleutherAI/pythia-160m-seed7
Updated • 105EleutherAI/pythia-160m-seed8
Updated • 105EleutherAI/pythia-160m-seed9
Updated • 104EleutherAI/pythia-410m
Text Generation • 0.5B • Updated • 112k • 31EleutherAI/pythia-410m-seed1
Updated • 775EleutherAI/pythia-410m-seed2
Updated • 569EleutherAI/pythia-410m-seed3
Updated • 571EleutherAI/pythia-410m-seed4
Updated • 571EleutherAI/pythia-410m-seed5
Updated • 563EleutherAI/pythia-410m-seed6
Updated • 109EleutherAI/pythia-410m-seed7
Updated • 107EleutherAI/pythia-410m-seed8
Updated • 108EleutherAI/pythia-410m-seed9
Updated • 109
EleutherAI/pythia-160m-data-seed1
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-data-seed2
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-data-seed3
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the data seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed1
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed2
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
EleutherAI/pythia-160m-weight-seed3
Updated • 2Note Version where the data order and weight initialization seeds are decoupled. Here, only the weight initialization seed is different from pythia-160m ("seed 0").
PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs
Paper • 2503.09543 • Published