FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper
•
2506.20920
•
Published
•
23
📝 Papers
Hugging Face Science team papers
-
-
SmolVLM: Redefining small and efficient multimodal models
Paper • 2504.05299 • Published • 191 -
YourBench: Easy Custom Evaluation Sets for Everyone
Paper • 2504.01833 • Published • 21 -
SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Paper • 2502.02737 • Published • 235 -
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Paper • 2406.17557 • Published • 98 -
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
Paper • 2405.18392 • Published • 12 -
StarCoder 2 and The Stack v2: The Next Generation
Paper • 2402.19173 • Published • 147 -
GAIA: a benchmark for General AI Assistants
Paper • 2311.12983 • Published • 220 -
Zephyr: Direct Distillation of LM Alignment
Paper • 2310.16944 • Published • 122 -
Scaling Data-Constrained Language Models
Paper • 2305.16264 • Published • 17 -
StarCoder: may the source be with you!
Paper • 2305.06161 • Published • 31 -
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Paper • 2303.03915 • Published • 7 -
SantaCoder: don't reach for the stars!
Paper • 2301.03988 • Published • 7 -
The Stack: 3 TB of permissively licensed source code
Paper • 2211.15533 • Published • 5 -
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Paper • 2211.05100 • Published • 32