FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language Paper β’ 2506.20920 β’ Published 2 days ago β’ 23
Running 66 66 Scaling FineWeb to 1000+ languages: Step 1: finding signal in 100s of evaluation tasks π Evaluate multilingual models using FineTasks